[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:10 dist-test-slave-xz51 krb5kdc[26946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:10 dist-test-slave-xz51 krb5kdc[26946](info): set up 2 sockets
Sep 29 19:05:10 dist-test-slave-xz51 krb5kdc[26946](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 19:05:11.245626 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:11.365334 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:35479
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:35479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:11.447170 26959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:11.447419 26959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:11.447455 26959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:11.448591 26959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:11.448661 26959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:11.448685 26959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:11.448706 26959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:11.448736 26959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:11.450266 26959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:35479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:35479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:11.450541 26959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:11.450775 26959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:11.456195 26965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:11.456176 26964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:11.456174 26967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:11.456270 26959 server_base.cc:1034] running on GCE node
I20260929 19:05:11.456647 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:11.456950 26959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:11.458132 26959 hybrid_clock.cc:648] HybridClock initialized: now 1790708711458125 us; error 37 us; skew 500 ppm
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:11.460706 26959 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:11.461148 26959 webserver.cc:460] Webserver started at http://127.26.78.254:44105/ using document root <none> and password file <none>
I20260929 19:05:11.461376 26959 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:11.461445 26959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:11.461598 26959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:11.462721 26959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "72fc5c31894a4cd58fca0096f81d1e4b"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "850bc5e88d39dc67a04ecedf7bf748e7"
server_key_iv: "e9e541082ca395cc9291c1786b355a39"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.463125 26959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "72fc5c31894a4cd58fca0096f81d1e4b"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "850bc5e88d39dc67a04ecedf7bf748e7"
server_key_iv: "e9e541082ca395cc9291c1786b355a39"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.464430 26959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:11.465159 26974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:11.465345 26959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:11.465443 26959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "72fc5c31894a4cd58fca0096f81d1e4b"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "850bc5e88d39dc67a04ecedf7bf748e7"
server_key_iv: "e9e541082ca395cc9291c1786b355a39"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.465531 26959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:11.482004 26959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:11.483949 26959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:11.484119 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:11.488178 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:35479
I20260929 19:05:11.488216 27026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:35479 every 8 connection(s)
I20260929 19:05:11.488576 26959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:11.489463 27027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:11.490161 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 26959
I20260929 19:05:11.490271 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:11.490976 26939 external_mini_cluster.cc:1428] Setting key af21efc2a713f64d8a64e4f551dd62cd
I20260929 19:05:11.492370 27027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b: Bootstrap starting.
I20260929 19:05:11.492959 27027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:11.493251 27027 log.cc:824] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:11.493994 27027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b: No bootstrap required, opened a new log
I20260929 19:05:11.495610 27027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 35479 } }
I20260929 19:05:11.495769 27027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:11.495814 27027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fc5c31894a4cd58fca0096f81d1e4b, State: Initialized, Role: FOLLOWER
I20260929 19:05:11.495968 27027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 35479 } }
I20260929 19:05:11.496057 27027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:11.496100 27027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:11.496156 27027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:11.496953 27027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 35479 } }
I20260929 19:05:11.497078 27027 leader_election.cc:302] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72fc5c31894a4cd58fca0096f81d1e4b; no voters:
I20260929 19:05:11.497280 27027 leader_election.cc:288] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:11.497308 27032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:11.497412 27032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [term 1 LEADER]: Becoming Leader. State: Replica: 72fc5c31894a4cd58fca0096f81d1e4b, State: Running, Role: LEADER
I20260929 19:05:11.497493 27032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 35479 } }
I20260929 19:05:11.497532 27027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:11.497859 27033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72fc5c31894a4cd58fca0096f81d1e4b. Latest consensus state: current_term: 1 leader_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 35479 } } }
I20260929 19:05:11.497949 27033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:11.498162 27036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:11.498742 27036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:11.498919 27034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72fc5c31894a4cd58fca0096f81d1e4b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 35479 } } }
I20260929 19:05:11.498996 27034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:11.501634 27036 catalog_manager.cc:1347] Generated new cluster ID: 2d12ed641d27429597b4b39810bdddc1
I20260929 19:05:11.501699 27036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:11.509742 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:11.491943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44812 (local address 127.26.78.254:35479)
0929 19:05:11.492104 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:11.492115 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:11.492351 (+ 236us) server_negotiation.cc:407] Connection header received
0929 19:05:11.495589 (+ 3238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:11.495595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:11.495643 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:11.495934 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:11.497633 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.499420 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.500487 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.501083 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.501197 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:11.505564 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:11.505570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:11.505571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:11.505585 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:11.507659 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:11.508164 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:11.508168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:11.508168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:11.508298 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:11.508587 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:11.508588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:11.508588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:11.508918 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:11.509078 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:11.509362 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 19:05:11.509508 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260929 19:05:11.533546 27036 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:11.534183 27036 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:11.541218 27036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72fc5c31894a4cd58fca0096f81d1e4b: Generated new TSK 0
I20260929 19:05:11.541448 27036 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:11.569558 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:35479
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:11.651513 27053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:11.651747 27053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:11.651787 27053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:11.652935 27053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:11.653000 27053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:11.653047 27053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:11.654637 27053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:35479
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:11.654911 27053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:11.655177 27053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:11.660127 27059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:11.660184 27058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:11.660342 27053 server_base.cc:1034] running on GCE node
W20260929 19:05:11.660132 27061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:11.660604 27053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:11.660876 27053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:11.662041 27053 hybrid_clock.cc:648] HybridClock initialized: now 1790708711662028 us; error 44 us; skew 500 ppm
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:11.664525 27053 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:11.664966 27053 webserver.cc:460] Webserver started at http://127.26.78.193:45475/ using document root <none> and password file <none>
I20260929 19:05:11.665212 27053 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:11.665289 27053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:11.665441 27053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:11.666829 27053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "7e803415b0a7489ba8e6608aa16c46b1"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "fbc7be6573bf227a2419a330fe6fd8f5"
server_key_iv: "1c29802e1ca5be3e3e949abaf0d0015f"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.667204 27053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "7e803415b0a7489ba8e6608aa16c46b1"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "fbc7be6573bf227a2419a330fe6fd8f5"
server_key_iv: "1c29802e1ca5be3e3e949abaf0d0015f"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.668697 27053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:11.669484 27068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:11.669656 27053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:11.669764 27053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "7e803415b0a7489ba8e6608aa16c46b1"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "fbc7be6573bf227a2419a330fe6fd8f5"
server_key_iv: "1c29802e1ca5be3e3e949abaf0d0015f"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.669862 27053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:11.685909 27053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:11.687753 27053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:11.687919 27053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:11.688167 27053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:11.688486 27053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:11.688542 27053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:11.688583 27053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:11.688611 27053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:11.694802 27053 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:36985
I20260929 19:05:11.694849 27181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:36985 every 8 connection(s)
I20260929 19:05:11.695214 27053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:11.704939 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 27053
I20260929 19:05:11.705046 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:11.705214 26939 external_mini_cluster.cc:1428] Setting key d1ed944f599508500e33891ad445f2df
I20260929 19:05:11.705520 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:11.695960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:49075 (local address 127.26.78.254:35479)
0929 19:05:11.696022 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:11.696025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:11.696244 (+ 219us) server_negotiation.cc:407] Connection header received
0929 19:05:11.696791 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:11.696792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:11.696808 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:11.696870 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:11.697985 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.698509 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.699677 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.699908 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.699990 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:11.702441 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:11.702443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:11.702444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:11.702452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:11.704024 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:11.704465 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:11.704467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:11.704467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:11.704529 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:11.704784 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:11.704785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:11.704785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:11.704914 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:11.704990 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:11.705339 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 19:05:11.705428 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:05:11.705658 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:11.695954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:35479 (local address 127.26.78.193:49075)
0929 19:05:11.696112 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:11.696152 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:11.696681 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:11.696942 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:11.696946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:11.697515 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:11.697890 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:11.697895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.698616 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.698617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:11.699563 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:11.699566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.699997 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.699998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:11.700133 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:11.700572 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:11.700580 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:11.702329 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:11.704131 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:11.704138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:11.704141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:11.704373 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:11.704600 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:11.704601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:11.704601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:11.704714 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:11.704982 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:11.704989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:11.705144 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 19:05:11.705260 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 19:05:11.705432 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260929 19:05:11.706347 27182 heartbeater.cc:347] Connected to a master server at 127.26.78.254:35479
I20260929 19:05:11.706453 27182 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:11.706705 27182 heartbeater.cc:511] Master 127.26.78.254:35479 requested a full tablet report, sending...
I20260929 19:05:11.707293 26991 ts_manager.cc:194] Registered new tserver with Master: 7e803415b0a7489ba8e6608aa16c46b1 (127.26.78.193:36985)
I20260929 19:05:11.707913 26991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:49075
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:11.731073 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:35479
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:11.812318 27187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:11.812528 27187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:11.812551 27187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:11.813728 27187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:11.813771 27187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:11.813798 27187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:11.815276 27187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:35479
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:11.815541 27187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:11.815781 27187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:11.821053 27192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:11.821063 27195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:11.821033 27193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:11.821273 27187 server_base.cc:1034] running on GCE node
I20260929 19:05:11.821455 27187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:11.821700 27187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:11.822862 27187 hybrid_clock.cc:648] HybridClock initialized: now 1790708711822833 us; error 46 us; skew 500 ppm
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:11.826056 27187 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:11.826550 27187 webserver.cc:460] Webserver started at http://127.26.78.194:33635/ using document root <none> and password file <none>
I20260929 19:05:11.826817 27187 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:11.826889 27187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:11.827050 27187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:11.828150 27187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "6a970697bfbc46d9b2e3c3f820615ff6"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "3249d63b276eb4a9cd4a1b2f64f8bc77"
server_key_iv: "2126006dab44f7654038b5cc28e2f152"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.828469 27187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "6a970697bfbc46d9b2e3c3f820615ff6"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "3249d63b276eb4a9cd4a1b2f64f8bc77"
server_key_iv: "2126006dab44f7654038b5cc28e2f152"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.829739 27187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:05:11.830492 27202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:11.830675 27187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:11.830773 27187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "6a970697bfbc46d9b2e3c3f820615ff6"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "3249d63b276eb4a9cd4a1b2f64f8bc77"
server_key_iv: "2126006dab44f7654038b5cc28e2f152"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.830856 27187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:11.850026 27187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:11.852698 27187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:11.852885 27187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:11.853161 27187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:11.853497 27187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:11.853559 27187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:11.853780 27187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:11.853940 27187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:11.860347 27187 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:46251
I20260929 19:05:11.860388 27315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:46251 every 8 connection(s)
I20260929 19:05:11.860728 27187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:11.866132 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 27187
I20260929 19:05:11.866240 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:11.866421 26939 external_mini_cluster.cc:1428] Setting key 1863fc110d449e83e76031054ed2965d
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:11.871501 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:11.861439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:60023 (local address 127.26.78.254:35479)
0929 19:05:11.861509 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:11.861511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:11.861777 (+ 266us) server_negotiation.cc:407] Connection header received
0929 19:05:11.862339 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:11.862340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:11.862352 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:11.862411 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:11.863844 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.864362 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.865415 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.865633 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.865723 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:11.868404 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:11.868405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:11.868406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:11.868422 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:11.870007 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:11.870467 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:11.870469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:11.870469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:11.870530 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:11.870810 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:11.870811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:11.870812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:11.870933 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:11.871005 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:11.871311 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 19:05:11.871409 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:11.871711 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:11.861414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:35479 (local address 127.26.78.194:60023)
0929 19:05:11.861598 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:11.861668 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:11.862225 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:11.862494 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:11.862497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:11.863348 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:11.863716 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:11.863724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.864461 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.864462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:11.865302 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:11.865305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:11.865732 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:11.865733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:11.865931 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:11.866494 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:11.866505 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:11.868282 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:11.870112 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:11.870120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:11.870122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:11.870371 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:11.870602 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:11.870603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:11.870603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:11.870724 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:11.871008 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:11.871015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:11.871165 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 19:05:11.871271 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 19:05:11.871455 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260929 19:05:11.872217 27316 heartbeater.cc:347] Connected to a master server at 127.26.78.254:35479
I20260929 19:05:11.872323 27316 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:11.872567 27316 heartbeater.cc:511] Master 127.26.78.254:35479 requested a full tablet report, sending...
I20260929 19:05:11.873003 26991 ts_manager.cc:194] Registered new tserver with Master: 6a970697bfbc46d9b2e3c3f820615ff6 (127.26.78.194:46251)
I20260929 19:05:11.873548 26991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:60023
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:11.894062 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:35479
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:11.976796 27321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:11.977025 27321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:11.977048 27321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:11.978413 27321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:11.978456 27321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:11.978484 27321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:11.979897 27321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:35479
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:11.980083 27321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:11.980270 27321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:11.985261 27326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:11.985268 27329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:11.985492 27321 server_base.cc:1034] running on GCE node
W20260929 19:05:11.985261 27327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:11.985746 27321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:11.986040 27321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:11.987212 27321 hybrid_clock.cc:648] HybridClock initialized: now 1790708711987192 us; error 37 us; skew 500 ppm
Sep 29 19:05:11 dist-test-slave-xz51 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:11.995574 27321 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:11.995994 27321 webserver.cc:460] Webserver started at http://127.26.78.195:35635/ using document root <none> and password file <none>
I20260929 19:05:11.996181 27321 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:11.996222 27321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:11.996318 27321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:11.997424 27321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "27d7d7e0cef64d06b60b4e734d91bce0"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "e123b031eb3533d93e5cf35c2d305389"
server_key_iv: "d493dbc66bc659bb9d8b7e783a7970f6"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.997753 27321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "27d7d7e0cef64d06b60b4e734d91bce0"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "e123b031eb3533d93e5cf35c2d305389"
server_key_iv: "d493dbc66bc659bb9d8b7e783a7970f6"
server_key_version: "encryptionkey@0"
I20260929 19:05:11.999317 27321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:12.000193 27336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:12.000382 27321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:12.000494 27321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "27d7d7e0cef64d06b60b4e734d91bce0"
format_stamp: "Formatted at 2026-09-29 19:05:11 on dist-test-slave-xz51"
server_key: "e123b031eb3533d93e5cf35c2d305389"
server_key_iv: "d493dbc66bc659bb9d8b7e783a7970f6"
server_key_version: "encryptionkey@0"
I20260929 19:05:12.000571 27321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:12.016215 27321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:12.018607 27321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:12.018772 27321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:12.019024 27321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:12.019344 27321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:12.019399 27321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:12.019443 27321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:12.019465 27321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:12.025609 27321 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:45369
I20260929 19:05:12.025695 27449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:45369 every 8 connection(s)
I20260929 19:05:12.026048 27321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:05:12.029014 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 27321
I20260929 19:05:12.029083 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:12.029243 26939 external_mini_cluster.cc:1428] Setting key cb099a1bc11f19f31476d976071a79a3
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[26946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708711, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:12.035920 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:12.026886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:45911 (local address 127.26.78.254:35479)
0929 19:05:12.026934 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:12.026936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:12.027138 (+ 202us) server_negotiation.cc:407] Connection header received
0929 19:05:12.027637 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:12.027638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:12.027649 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:12.027715 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:12.028638 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.029172 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.030611 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.030806 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.030860 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:12.033113 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:12.033114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:12.033115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:12.033121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:12.034733 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:12.035142 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:12.035143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:12.035143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:12.035197 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:12.035416 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:12.035417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:12.035417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:12.035521 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:12.035572 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:12.035792 (+ 220us) server_negotiation.cc:299] Negotiation successful
0929 19:05:12.035855 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:05:12.036175 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:12.026846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:35479 (local address 127.26.78.195:45911)
0929 19:05:12.027007 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:12.027050 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:12.027546 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:12.027779 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:12.027783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:12.028356 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:12.028556 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:12.028561 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.029245 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.029246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:12.030519 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:12.030523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.030881 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.030881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:12.030995 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:12.031495 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:12.031503 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:12.033018 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:12.034832 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:12.034838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:12.034841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:12.035074 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:12.035257 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:12.035258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:12.035258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:12.035359 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:12.035576 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:12.035581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:12.035696 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 19:05:12.035803 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 19:05:12.035926 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260929 19:05:12.036710 27450 heartbeater.cc:347] Connected to a master server at 127.26.78.254:35479
I20260929 19:05:12.036820 27450 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:12.036978 27450 heartbeater.cc:511] Master 127.26.78.254:35479 requested a full tablet report, sending...
I20260929 19:05:12.037333 26991 ts_manager.cc:194] Registered new tserver with Master: 27d7d7e0cef64d06b60b4e734d91bce0 (127.26.78.195:45369)
I20260929 19:05:12.037669 26991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:45911
I20260929 19:05:12.041489 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708712, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[26946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708712, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:12.065436 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:12.057075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.26.78.193:36985)
0929 19:05:12.057278 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:12.057283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:12.057295 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:05:12.057353 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:12.057355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:12.057370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:12.057440 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:12.058050 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.058791 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.059538 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.059878 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.059933 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:12.062645 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:12.062655 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:12.062655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:12.062662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:12.064167 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:12.064555 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:12.064559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:12.064559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:12.064655 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:12.064868 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:12.064869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:12.064869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:12.065021 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:12.065169 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:12.065207 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 19:05:12.065280 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":122,"threads_started":1}
W20260929 19:05:12.065950 27096 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33696
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[26946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:12.088176 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:12.081347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.26.78.193:36985)
0929 19:05:12.081418 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:12.081421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:12.081512 (+ 91us) server_negotiation.cc:407] Connection header received
0929 19:05:12.081563 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:12.081564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:12.081576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:12.081628 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:12.082281 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.082762 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.083453 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:12.083647 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:12.083699 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:12.085899 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:12.085901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:12.085901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:12.085908 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:12.087302 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:12.087585 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:12.087586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:12.087586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:12.087635 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:12.087777 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:12.087777 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:12.087778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:12.087863 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:12.087944 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:12.088015 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 19:05:12.088074 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:05:12.088748 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 27053
I20260929 19:05:12.093472 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 27187
I20260929 19:05:12.097889 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 27321
I20260929 19:05:12.102411 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 26959
2026-09-29T19:05:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1903 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[27464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[27464](info): set up 2 sockets
Sep 29 19:05:12 dist-test-slave-xz51 krb5kdc[27464](info): commencing operation
krb5kdc: starting...
W20260929 19:05:13.141407 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:13.260816 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44873
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:13.342633 27477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:13.342839 27477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:13.342864 27477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:13.344400 27477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:13.344467 27477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:13.344487 27477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:13.344504 27477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:13.344560 27477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:13.346280 27477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:13.346567 27477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:13.346830 27477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:13.352231 27482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:13.352269 27483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:13.352264 27477 server_base.cc:1034] running on GCE node
W20260929 19:05:13.352228 27485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:13.352804 27477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:13.353055 27477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:13.354194 27477 hybrid_clock.cc:648] HybridClock initialized: now 1790708713354170 us; error 39 us; skew 500 ppm
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:13.356685 27477 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:13.357139 27477 webserver.cc:460] Webserver started at http://127.26.78.254:39499/ using document root <none> and password file <none>
I20260929 19:05:13.357385 27477 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:13.357430 27477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:13.357590 27477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:13.358731 27477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "38fb95e1ebb744d89e9a23de5a909cab"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "bddf5178630c3674627f83722446d417"
server_key_iv: "9d2e1ef88ad55e69cfa9115c19dbdd4b"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.359105 27477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "38fb95e1ebb744d89e9a23de5a909cab"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "bddf5178630c3674627f83722446d417"
server_key_iv: "9d2e1ef88ad55e69cfa9115c19dbdd4b"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.360426 27477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:13.361169 27492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.361352 27477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:13.361438 27477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "38fb95e1ebb744d89e9a23de5a909cab"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "bddf5178630c3674627f83722446d417"
server_key_iv: "9d2e1ef88ad55e69cfa9115c19dbdd4b"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.361490 27477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:13.377242 27477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:13.379652 27477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:13.379763 27477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:13.383575 27477 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:44873
I20260929 19:05:13.383606 27544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:44873 every 8 connection(s)
I20260929 19:05:13.383972 27477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:13.384850 27545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:13.385599 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 27477
I20260929 19:05:13.385694 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:13.385896 26939 external_mini_cluster.cc:1428] Setting key 97f57b5249261c5e4855a9580e6cfe3d
I20260929 19:05:13.387678 27545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab: Bootstrap starting.
I20260929 19:05:13.388252 27545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:13.388551 27545 log.cc:824] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:13.389212 27545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab: No bootstrap required, opened a new log
I20260929 19:05:13.391018 27545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fb95e1ebb744d89e9a23de5a909cab" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44873 } }
I20260929 19:05:13.391177 27545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:13.391223 27545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38fb95e1ebb744d89e9a23de5a909cab, State: Initialized, Role: FOLLOWER
I20260929 19:05:13.391402 27545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fb95e1ebb744d89e9a23de5a909cab" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44873 } }
I20260929 19:05:13.391500 27545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:13.391544 27545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:13.391593 27545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:13.392097 27545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fb95e1ebb744d89e9a23de5a909cab" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44873 } }
I20260929 19:05:13.392213 27545 leader_election.cc:302] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38fb95e1ebb744d89e9a23de5a909cab; no voters:
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:13.392417 27545 leader_election.cc:288] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:13.392449 27550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:13.392566 27550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [term 1 LEADER]: Becoming Leader. State: Replica: 38fb95e1ebb744d89e9a23de5a909cab, State: Running, Role: LEADER
I20260929 19:05:13.392678 27545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:13.392741 27550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fb95e1ebb744d89e9a23de5a909cab" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44873 } }
I20260929 19:05:13.393110 27550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38fb95e1ebb744d89e9a23de5a909cab. Latest consensus state: current_term: 1 leader_uuid: "38fb95e1ebb744d89e9a23de5a909cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fb95e1ebb744d89e9a23de5a909cab" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44873 } } }
I20260929 19:05:13.393121 27552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38fb95e1ebb744d89e9a23de5a909cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fb95e1ebb744d89e9a23de5a909cab" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44873 } } }
I20260929 19:05:13.393252 27550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:13.393500 27552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab [sys.catalog]: This master's current role is: LEADER
W20260929 19:05:13.394558 27566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:05:13.394644 27566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:05:13.394722 27559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:13.394871 27559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:13.396396 27559 catalog_manager.cc:1347] Generated new cluster ID: 862c5d14664346ada885007d179f0f42
I20260929 19:05:13.396453 27559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:13.397176 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.386421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.26.78.254:44873)
0929 19:05:13.386629 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.386637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.386663 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:05:13.387063 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.387070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.387115 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.387264 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:13.387887 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.388884 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.389712 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.390195 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.390350 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.392953 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:13.392957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:13.392958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:13.392973 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:13.395179 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.395589 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.395592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.395593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.395699 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.395948 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.395949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.395949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.396256 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:13.396779 (+ 523us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.396858 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.396986 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260929 19:05:13.418185 27559 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:13.418820 27559 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:13.436774 27559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38fb95e1ebb744d89e9a23de5a909cab: Generated new TSK 0
I20260929 19:05:13.437009 27559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:13.468001 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44873
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:13.551136 27571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:13.551358 27571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:13.551384 27571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:13.552578 27571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:13.552624 27571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:13.552652 27571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:13.554167 27571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:13.554350 27571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:13.554519 27571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:13.559514 27577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:13.559510 27576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:13.559648 27571 server_base.cc:1034] running on GCE node
W20260929 19:05:13.559589 27579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:13.560009 27571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:13.560271 27571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:13.561443 27571 hybrid_clock.cc:648] HybridClock initialized: now 1790708713561425 us; error 42 us; skew 500 ppm
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:13.564023 27571 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:13.564507 27571 webserver.cc:460] Webserver started at http://127.26.78.193:46669/ using document root <none> and password file <none>
I20260929 19:05:13.564739 27571 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:13.564792 27571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:13.564927 27571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:13.566111 27571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "2d30aa4a85b841aabccd6e30365a8fea"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "c25effb2a1f44471856e102f3c51896e"
server_key_iv: "c042f199d171b6266b898ee601b5b30d"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.566468 27571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "2d30aa4a85b841aabccd6e30365a8fea"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "c25effb2a1f44471856e102f3c51896e"
server_key_iv: "c042f199d171b6266b898ee601b5b30d"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.567876 27571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:13.568667 27586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.568887 27571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:13.568995 27571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "2d30aa4a85b841aabccd6e30365a8fea"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "c25effb2a1f44471856e102f3c51896e"
server_key_iv: "c042f199d171b6266b898ee601b5b30d"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.569069 27571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:13.584349 27571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:13.586809 27571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:13.586975 27571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:13.587209 27571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:13.587538 27571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:13.587592 27571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.587635 27571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:13.587662 27571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.593669 27571 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:38203
I20260929 19:05:13.593763 27699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:38203 every 8 connection(s)
I20260929 19:05:13.594099 27571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:13.602993 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 27571
I20260929 19:05:13.603092 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:13.603310 26939 external_mini_cluster.cc:1428] Setting key e874d5988bde6e5baf443a05167ba344
I20260929 19:05:13.603837 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.594872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:33049 (local address 127.26.78.254:44873)
0929 19:05:13.594934 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.594937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.595148 (+ 211us) server_negotiation.cc:407] Connection header received
0929 19:05:13.596043 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.596045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.596054 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.596122 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:13.596684 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.597189 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.598323 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.598556 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.598629 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.600823 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:13.600825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:13.600825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:13.600832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:13.602237 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.602675 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.602677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.602677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.602741 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.603081 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.603082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.603082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.603223 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:13.603278 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.603644 (+ 366us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.603743 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:05:13.604071 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.594858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44873 (local address 127.26.78.193:33049)
0929 19:05:13.595021 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:13.595062 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:13.595956 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:13.596173 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:13.596177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:13.596418 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:13.596605 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.596609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.597273 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.597274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.598218 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.598220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.598636 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.598637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.598735 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.599308 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:13.599317 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:13.600673 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:13.602321 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:13.602328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:13.602330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:13.602563 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:13.602828 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:13.602828 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:13.602829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:13.602983 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:13.603294 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:13.603300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:13.603479 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 19:05:13.603612 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 19:05:13.603811 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260929 19:05:13.604581 27700 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44873
I20260929 19:05:13.604696 27700 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:13.604900 27700 heartbeater.cc:511] Master 127.26.78.254:44873 requested a full tablet report, sending...
I20260929 19:05:13.605413 27509 ts_manager.cc:194] Registered new tserver with Master: 2d30aa4a85b841aabccd6e30365a8fea (127.26.78.193:38203)
I20260929 19:05:13.606138 27509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:33049
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:13.628703 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44873
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:13.708518 27705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:13.708714 27705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:13.708737 27705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:13.709805 27705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:13.709861 27705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:13.709887 27705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:13.711266 27705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:13.711472 27705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:13.711634 27705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:13.716682 27711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:13.716655 27713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:13.716722 27710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:13.716881 27705 server_base.cc:1034] running on GCE node
I20260929 19:05:13.717078 27705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:13.717368 27705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:13.718528 27705 hybrid_clock.cc:648] HybridClock initialized: now 1790708713718522 us; error 36 us; skew 500 ppm
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:13.720851 27705 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:13.721305 27705 webserver.cc:460] Webserver started at http://127.26.78.194:42179/ using document root <none> and password file <none>
I20260929 19:05:13.721530 27705 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:13.721599 27705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:13.721757 27705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:13.722860 27705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "a439734d0912462bbadc7207a8563610"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "16936b7386e0efaea5ab65f94ed5bfb3"
server_key_iv: "ac35189e201fe6f793ec1f7e9d3ac86d"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.723223 27705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "a439734d0912462bbadc7207a8563610"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "16936b7386e0efaea5ab65f94ed5bfb3"
server_key_iv: "ac35189e201fe6f793ec1f7e9d3ac86d"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.724519 27705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:13.725251 27720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.725435 27705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:13.725533 27705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "a439734d0912462bbadc7207a8563610"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "16936b7386e0efaea5ab65f94ed5bfb3"
server_key_iv: "ac35189e201fe6f793ec1f7e9d3ac86d"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.725618 27705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:13.740374 27705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:13.742373 27705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:13.742545 27705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:13.742794 27705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:13.743117 27705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:13.743170 27705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.743206 27705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:13.743233 27705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.749737 27705 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39229
I20260929 19:05:13.749780 27833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39229 every 8 connection(s)
I20260929 19:05:13.750157 27705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:13.753545 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 27705
I20260929 19:05:13.753618 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:13.753803 26939 external_mini_cluster.cc:1428] Setting key 3cb94159accac5848f814fd364ff9599
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:13.760823 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.751048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:43373 (local address 127.26.78.254:44873)
0929 19:05:13.751142 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.751144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.751320 (+ 176us) server_negotiation.cc:407] Connection header received
0929 19:05:13.752308 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.752310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.752324 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.752370 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:13.753032 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.753557 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.754990 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.755189 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.755283 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.757795 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:13.757796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:13.757797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:13.757805 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:13.759461 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.759856 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.759858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.759858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.759924 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.760154 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.760155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.760156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.760311 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:13.760384 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.760627 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.760714 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 19:05:13.760896 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.750935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44873 (local address 127.26.78.194:43373)
0929 19:05:13.751179 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:13.751228 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:13.752175 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:13.752458 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:13.752461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:13.752716 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:13.752931 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.752935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.753709 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.753709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.754899 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.754901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.755284 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.755285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.755420 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.756107 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:13.756116 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:13.757703 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:13.759553 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:13.759559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:13.759561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:13.759775 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:13.759996 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:13.759997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:13.759997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:13.760090 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:13.760377 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:13.760381 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:13.760495 (+ 114us) client_negotiation.cc:769] Sending connection context
0929 19:05:13.760578 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 19:05:13.760696 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":122,"threads_started":1}
I20260929 19:05:13.761379 27834 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44873
I20260929 19:05:13.761475 27834 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:13.761615 27834 heartbeater.cc:511] Master 127.26.78.254:44873 requested a full tablet report, sending...
I20260929 19:05:13.762008 27509 ts_manager.cc:194] Registered new tserver with Master: a439734d0912462bbadc7207a8563610 (127.26.78.194:39229)
I20260929 19:05:13.762543 27509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:43373
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:13.781637 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44873
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:13.863564 27839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:13.863809 27839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:13.863855 27839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:13.865046 27839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:13.865119 27839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:13.865170 27839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:13.866896 27839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:13.867182 27839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:13.867422 27839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:13.872560 27844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:13.872574 27847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:13.872664 27839 server_base.cc:1034] running on GCE node
W20260929 19:05:13.872668 27845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:13.873008 27839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:13.873210 27839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:13.874344 27839 hybrid_clock.cc:648] HybridClock initialized: now 1790708713874323 us; error 38 us; skew 500 ppm
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:13.876834 27839 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:13.877226 27839 webserver.cc:460] Webserver started at http://127.26.78.195:43957/ using document root <none> and password file <none>
I20260929 19:05:13.877434 27839 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:13.877475 27839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:13.877563 27839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:13.878651 27839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "20170f5bd0f84227a541aa85bfed4363"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "489230fe403eebd5e0edc5ad208b46e7"
server_key_iv: "443ae24ae5c6419dbae8c18839e50cdb"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.878947 27839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "20170f5bd0f84227a541aa85bfed4363"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "489230fe403eebd5e0edc5ad208b46e7"
server_key_iv: "443ae24ae5c6419dbae8c18839e50cdb"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.880250 27839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:05:13.880937 27854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.881122 27839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:13.881222 27839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "20170f5bd0f84227a541aa85bfed4363"
format_stamp: "Formatted at 2026-09-29 19:05:13 on dist-test-slave-xz51"
server_key: "489230fe403eebd5e0edc5ad208b46e7"
server_key_iv: "443ae24ae5c6419dbae8c18839e50cdb"
server_key_version: "encryptionkey@0"
I20260929 19:05:13.881327 27839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:13.900868 27839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:13.902995 27839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:13.903165 27839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:13.903434 27839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:13.903764 27839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:13.903860 27839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.903934 27839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:13.903966 27839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:13.909683 27839 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:38037
I20260929 19:05:13.909758 27967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:38037 every 8 connection(s)
I20260929 19:05:13.910123 27839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:05:13 dist-test-slave-xz51 krb5kdc[27464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:13.916615 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 27839
I20260929 19:05:13.916687 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:13.916875 26939 external_mini_cluster.cc:1428] Setting key 62b81ad46a14c1ffcac7ef870aa16ccd
I20260929 19:05:13.919539 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.910825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:46275 (local address 127.26.78.254:44873)
0929 19:05:13.910898 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.910900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.911096 (+ 196us) server_negotiation.cc:407] Connection header received
0929 19:05:13.911995 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.911997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.912007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.912058 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:13.912621 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.913109 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.914190 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.914376 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.914431 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.916412 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:13.916414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:13.916414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:13.916420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:13.918349 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.918735 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.918736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.918736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.918797 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.919006 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.919007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.919007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.919109 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:13.919161 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.919413 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.919475 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:05:13.919699 27970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.910826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44873 (local address 127.26.78.195:46275)
0929 19:05:13.910976 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:13.911012 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:13.911907 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:13.912115 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:13.912118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:13.912366 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:13.912555 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.912558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.913183 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.913183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.914117 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.914120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.914448 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.914449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.914553 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.914995 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:13.915003 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:13.916325 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:13.918433 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:13.918439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:13.918440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:13.918658 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:13.918861 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:13.918861 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:13.918862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:13.918953 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:13.919163 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:13.919167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:13.919278 (+ 111us) client_negotiation.cc:769] Sending connection context
0929 19:05:13.919384 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 19:05:13.919501 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260929 19:05:13.920141 27968 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44873
I20260929 19:05:13.920256 27968 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:13.920447 27968 heartbeater.cc:511] Master 127.26.78.254:44873 requested a full tablet report, sending...
I20260929 19:05:13.920823 27509 ts_manager.cc:194] Registered new tserver with Master: 20170f5bd0f84227a541aa85bfed4363 (127.26.78.195:38037)
I20260929 19:05:13.921197 27509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:46275
I20260929 19:05:13.928539 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:05:13.935973 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.930309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.26.78.254:44873)
0929 19:05:13.930385 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.930387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.930414 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:05:13.930482 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.930483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.930492 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.930545 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:13.931048 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.931526 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.932207 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.932392 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.932443 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.933560 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:13.933561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:13.933561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:13.933567 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:13.934999 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.935300 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.935300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.935301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.935350 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:13.935509 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:13.935509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:13.935510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:13.935644 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:13.935715 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.935812 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.935868 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 19:05:13.938429 27509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:13.939980 27509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:13.946872 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.942567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:38037 (local address 127.0.0.1:56702)
0929 19:05:13.942722 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:13.942764 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:13.942856 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:13.943122 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:13.943125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:13.943127 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:13.943203 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.943206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.944564 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.944565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.945874 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.945877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.946568 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.946569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.946653 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.946666 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 19:05:13.946737 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 19:05:13.946794 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260929 19:05:13.946882 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.942658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56702 (local address 127.26.78.195:38037)
0929 19:05:13.942942 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.942948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.942959 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:05:13.943004 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.943007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.943024 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.943111 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:13.943322 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.944462 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.945949 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.946474 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.946570 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.946630 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.946761 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.946804 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":83,"threads_started":1}
I20260929 19:05:13.947168 27979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.942579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39229 (local address 127.0.0.1:47554)
0929 19:05:13.943196 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:13.943206 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:13.943259 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:13.943505 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:13.943506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:13.943508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:13.943562 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.943563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.944737 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.944738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.945897 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.945898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.946870 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.946871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.946936 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.946987 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 19:05:13.947033 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:05:13.947071 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260929 19:05:13.947455 27984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.942942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.26.78.194:39229)
0929 19:05:13.943153 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.943158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.943331 (+ 173us) server_negotiation.cc:407] Connection header received
0929 19:05:13.943376 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.943378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.943395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.943468 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:13.943631 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.944638 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.945988 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.946718 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.946788 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.946836 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.947337 (+ 501us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.947375 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
I20260929 19:05:13.947804 27902 tablet_service.cc:1467] Processing CreateTablet for tablet 301e366f5f6d4b2cbe37872b37172e91 (DEFAULT_TABLE table=test-table [id=93c2b5e3d9584645a60e39b152646d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:13.948165 27902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 301e366f5f6d4b2cbe37872b37172e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:13.948222 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.942789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:38203 (local address 127.0.0.1:50272)
0929 19:05:13.943211 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:13.943220 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:13.943282 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:13.944857 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:13.944858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:13.944860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:13.944902 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.944904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.945998 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.945999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.947077 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:13.947079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.947998 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.947999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:13.948058 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.948062 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:13.948119 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 19:05:13.948157 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":68,"threads_started":1}
I20260929 19:05:13.948240 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:13.942930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.26.78.193:38203)
0929 19:05:13.944620 (+ 1690us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:13.944625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:13.944635 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:13.944674 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:13.944676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:13.944693 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:13.944764 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:13.944984 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.945910 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.947161 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:13.947915 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:13.947998 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:13.948051 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:13.948132 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 19:05:13.948169 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1646,"thread_start_us":88,"threads_started":1}
I20260929 19:05:13.948872 27634 tablet_service.cc:1467] Processing CreateTablet for tablet 301e366f5f6d4b2cbe37872b37172e91 (DEFAULT_TABLE table=test-table [id=93c2b5e3d9584645a60e39b152646d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:13.949162 27634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 301e366f5f6d4b2cbe37872b37172e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:13.951210 27985 tablet_bootstrap.cc:492] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363: Bootstrap starting.
I20260929 19:05:13.951656 27768 tablet_service.cc:1467] Processing CreateTablet for tablet 301e366f5f6d4b2cbe37872b37172e91 (DEFAULT_TABLE table=test-table [id=93c2b5e3d9584645a60e39b152646d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:13.951982 27768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 301e366f5f6d4b2cbe37872b37172e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:13.952028 27985 tablet_bootstrap.cc:654] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:13.952018 27986 tablet_bootstrap.cc:492] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea: Bootstrap starting.
I20260929 19:05:13.952555 27986 tablet_bootstrap.cc:654] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:13.952617 27985 log.cc:824] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:13.953140 27986 log.cc:824] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:13.953472 27985 tablet_bootstrap.cc:492] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363: No bootstrap required, opened a new log
I20260929 19:05:13.953548 27985 ts_tablet_manager.cc:1397] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:05:13.953876 27986 tablet_bootstrap.cc:492] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea: No bootstrap required, opened a new log
I20260929 19:05:13.953961 27986 ts_tablet_manager.cc:1397] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:05:13.955313 27989 tablet_bootstrap.cc:492] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610: Bootstrap starting.
I20260929 19:05:13.955691 27985 raft_consensus.cc:348] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:13.955749 27986 raft_consensus.cc:348] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:13.955875 27985 raft_consensus.cc:374] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:13.955920 27985 raft_consensus.cc:729] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20170f5bd0f84227a541aa85bfed4363, State: Initialized, Role: FOLLOWER
I20260929 19:05:13.955924 27986 raft_consensus.cc:374] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:13.955964 27986 raft_consensus.cc:729] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d30aa4a85b841aabccd6e30365a8fea, State: Initialized, Role: FOLLOWER
I20260929 19:05:13.956077 27989 tablet_bootstrap.cc:654] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:13.956054 27985 consensus_queue.cc:260] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:13.956105 27986 consensus_queue.cc:260] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:13.956319 27985 ts_tablet_manager.cc:1428] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:05:13.956485 27968 heartbeater.cc:503] Master 127.26.78.254:44873 was elected leader, sending a full tablet report...
I20260929 19:05:13.956509 27700 heartbeater.cc:503] Master 127.26.78.254:44873 was elected leader, sending a full tablet report...
I20260929 19:05:13.956633 27989 log.cc:824] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:13.956955 27986 ts_tablet_manager.cc:1428] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:05:13.957646 27989 tablet_bootstrap.cc:492] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610: No bootstrap required, opened a new log
I20260929 19:05:13.957769 27989 ts_tablet_manager.cc:1397] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:05:13.959376 27989 raft_consensus.cc:348] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:13.959509 27989 raft_consensus.cc:374] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:13.959532 27989 raft_consensus.cc:729] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a439734d0912462bbadc7207a8563610, State: Initialized, Role: FOLLOWER
I20260929 19:05:13.959663 27989 consensus_queue.cc:260] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:13.959949 27989 ts_tablet_manager.cc:1428] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:05:13.959966 27834 heartbeater.cc:503] Master 127.26.78.254:44873 was elected leader, sending a full tablet report...
W20260929 19:05:14.000793 27835 tablet.cc:2369] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:05:14.014053 27993 raft_consensus.cc:482] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:14.014210 27993 raft_consensus.cc:504] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:14.014636 27993 leader_election.cc:288] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20170f5bd0f84227a541aa85bfed4363 (127.26.78.195:38037), 2d30aa4a85b841aabccd6e30365a8fea (127.26.78.193:38203)
I20260929 19:05:14.017925 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:14.014942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:44237 (local address 127.26.78.195:38037)
0929 19:05:14.014989 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:14.014992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:14.015006 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:05:14.015045 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:14.015046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:14.015056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:14.015141 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:14.015346 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.016065 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.017121 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.017522 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.017578 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:14.017585 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:14.017786 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 19:05:14.017851 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:14.017930 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:14.014871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:38037 (local address 127.26.78.194:44237)
0929 19:05:14.014934 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:14.014947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:14.014993 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:14.015185 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:14.015186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:14.015188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:14.015266 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:14.015268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.016151 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.016152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:14.016926 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:14.016928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.017611 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.017612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:14.017703 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:14.017706 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:14.017749 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 19:05:14.017842 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 19:05:14.017956 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:14.014951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:49973 (local address 127.26.78.193:38203)
0929 19:05:14.015004 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:14.015007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:14.015172 (+ 165us) server_negotiation.cc:407] Connection header received
0929 19:05:14.015204 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:14.015205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:14.015214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:14.015302 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:14.015454 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.016136 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.017169 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.017538 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.017634 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:14.017642 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:14.017849 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 19:05:14.017891 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:05:14.017948 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:14.014870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:38203 (local address 127.26.78.194:49973)
0929 19:05:14.015083 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:14.015093 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:14.015155 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:14.015286 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:14.015287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:14.015288 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:14.015382 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:14.015384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.016228 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.016228 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:14.016955 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:14.016957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.017663 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.017664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:14.017722 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:14.017725 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:14.017804 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 19:05:14.017864 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":18,"thread_start_us":116,"threads_started":1}
I20260929 19:05:14.018287 27922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301e366f5f6d4b2cbe37872b37172e91" candidate_uuid: "a439734d0912462bbadc7207a8563610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20170f5bd0f84227a541aa85bfed4363" is_pre_election: true
I20260929 19:05:14.018491 27922 raft_consensus.cc:2454] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a439734d0912462bbadc7207a8563610 in term 0.
I20260929 19:05:14.018573 27654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301e366f5f6d4b2cbe37872b37172e91" candidate_uuid: "a439734d0912462bbadc7207a8563610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d30aa4a85b841aabccd6e30365a8fea" is_pre_election: true
I20260929 19:05:14.018718 27654 raft_consensus.cc:2454] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a439734d0912462bbadc7207a8563610 in term 0.
I20260929 19:05:14.018734 27723 leader_election.cc:302] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20170f5bd0f84227a541aa85bfed4363, a439734d0912462bbadc7207a8563610; no voters:
I20260929 19:05:14.018859 27993 raft_consensus.cc:2781] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:14.018954 27993 raft_consensus.cc:482] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:14.018982 27993 raft_consensus.cc:3037] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:14.019706 27993 raft_consensus.cc:504] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:14.019855 27993 leader_election.cc:288] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [CANDIDATE]: Term 1 election: Requested vote from peers 20170f5bd0f84227a541aa85bfed4363 (127.26.78.195:38037), 2d30aa4a85b841aabccd6e30365a8fea (127.26.78.193:38203)
I20260929 19:05:14.020036 27922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301e366f5f6d4b2cbe37872b37172e91" candidate_uuid: "a439734d0912462bbadc7207a8563610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20170f5bd0f84227a541aa85bfed4363"
I20260929 19:05:14.020133 27922 raft_consensus.cc:3037] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:14.020145 27654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301e366f5f6d4b2cbe37872b37172e91" candidate_uuid: "a439734d0912462bbadc7207a8563610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d30aa4a85b841aabccd6e30365a8fea"
I20260929 19:05:14.020239 27654 raft_consensus.cc:3037] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:14.020859 27922 raft_consensus.cc:2454] T 301e366f5f6d4b2cbe37872b37172e91 P 20170f5bd0f84227a541aa85bfed4363 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a439734d0912462bbadc7207a8563610 in term 1.
I20260929 19:05:14.021023 27654 raft_consensus.cc:2454] T 301e366f5f6d4b2cbe37872b37172e91 P 2d30aa4a85b841aabccd6e30365a8fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a439734d0912462bbadc7207a8563610 in term 1.
I20260929 19:05:14.021052 27723 leader_election.cc:302] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20170f5bd0f84227a541aa85bfed4363, a439734d0912462bbadc7207a8563610; no voters:
I20260929 19:05:14.021199 27993 raft_consensus.cc:2781] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:14.021389 27993 raft_consensus.cc:686] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [term 1 LEADER]: Becoming Leader. State: Replica: a439734d0912462bbadc7207a8563610, State: Running, Role: LEADER
I20260929 19:05:14.021493 27993 consensus_queue.cc:237] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } }
I20260929 19:05:14.022464 27509 catalog_manager.cc:5495] T 301e366f5f6d4b2cbe37872b37172e91 P a439734d0912462bbadc7207a8563610 reported cstate change: term changed from 0 to 1, leader changed from <none> to a439734d0912462bbadc7207a8563610 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "a439734d0912462bbadc7207a8563610" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20170f5bd0f84227a541aa85bfed4363" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 38037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a439734d0912462bbadc7207a8563610" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d30aa4a85b841aabccd6e30365a8fea" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38203 } health_report { overall_health: UNKNOWN } } }
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[27464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:14.042661 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:14.035489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.26.78.193:38203)
0929 19:05:14.035567 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:14.035568 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:14.035633 (+ 65us) server_negotiation.cc:407] Connection header received
0929 19:05:14.035679 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:14.035680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:14.035690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:14.035739 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:14.036269 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.036738 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.037534 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.037715 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.037770 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:14.040064 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:14.040074 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:14.040075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:14.040081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:14.041603 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:14.041977 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:14.041980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:14.041980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:14.042033 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:14.042193 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:14.042194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:14.042194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:14.042331 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:14.042427 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:14.042488 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 19:05:14.042550 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[27464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708714, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[27464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708714, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:14.066131 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:14.059039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50290 (local address 127.26.78.193:38203)
0929 19:05:14.059109 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:14.059111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:14.059208 (+ 97us) server_negotiation.cc:407] Connection header received
0929 19:05:14.059273 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:14.059274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:14.059295 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:14.059340 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:14.059907 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.060379 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.061069 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.061245 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.061303 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:14.063563 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:14.063565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:14.063565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:14.063572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:14.064825 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:14.065188 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:14.065189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:14.065189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:14.065250 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:14.065448 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:14.065449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:14.065450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:14.065557 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:14.065639 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:14.065969 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 19:05:14.066043 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[27464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[27464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:14.088121 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:14.081368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50296 (local address 127.26.78.193:38203)
0929 19:05:14.081437 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:14.081439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:14.081512 (+ 73us) server_negotiation.cc:407] Connection header received
0929 19:05:14.081579 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:14.081579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:14.081588 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:14.081634 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:14.082223 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.082681 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.083382 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:14.083560 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:14.083609 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:14.086009 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:14.086010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:14.086011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:14.086018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:14.087186 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:14.087504 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:14.087505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:14.087506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:14.087555 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:14.087712 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:14.087713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:14.087713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:14.087808 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:14.087881 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:14.087963 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 19:05:14.088018 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260929 19:05:14.088630 27614 tablet_service.cc:2908] Rejecting scan request for tablet 301e366f5f6d4b2cbe37872b37172e91: Uninitialized: clean time has not yet been initialized
I20260929 19:05:14.089715 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 27571
I20260929 19:05:14.095306 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 27705
I20260929 19:05:14.099881 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 27839
I20260929 19:05:14.104889 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 27477
2026-09-29T19:05:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2003 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[28009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[28009](info): set up 2 sockets
Sep 29 19:05:14 dist-test-slave-xz51 krb5kdc[28009](info): commencing operation
krb5kdc: starting...
W20260929 19:05:15.205570 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:15.325724 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:46561
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:46561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:15.405437 28022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:15.405711 28022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:15.405748 28022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:15.406894 28022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:15.406968 28022 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 19:05:15.406989 28022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:15.407011 28022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:15.407042 28022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:15.407065 28022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:15.408528 28022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:46561
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:46561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:15.408807 28022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:15.409036 28022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:15.414065 28030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:15.414112 28022 server_base.cc:1034] running on GCE node
W20260929 19:05:15.414095 28027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:15.414095 28028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:15.414595 28022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:15.414873 28022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:15.416015 28022 hybrid_clock.cc:648] HybridClock initialized: now 1790708715415994 us; error 41 us; skew 500 ppm
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:15.418551 28022 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:15.419013 28022 webserver.cc:460] Webserver started at http://127.26.78.254:39829/ using document root <none> and password file <none>
I20260929 19:05:15.419260 28022 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:15.419329 28022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:15.419483 28022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:15.420564 28022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "9fdae13cdc20493e838da4c08233c9e4"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "6f593cecc9adcf72b44d7f45f5329db0"
server_key_iv: "4ee4b28402176b43c48ab7da451cc78f"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.420933 28022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "9fdae13cdc20493e838da4c08233c9e4"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "6f593cecc9adcf72b44d7f45f5329db0"
server_key_iv: "4ee4b28402176b43c48ab7da451cc78f"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.422309 28022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:05:15.423000 28037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.423177 28022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:15.423285 28022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "9fdae13cdc20493e838da4c08233c9e4"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "6f593cecc9adcf72b44d7f45f5329db0"
server_key_iv: "4ee4b28402176b43c48ab7da451cc78f"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.423359 28022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:15.440367 28022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:15.442917 28022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:15.443089 28022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:15.447625 28022 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:46561
I20260929 19:05:15.447780 28099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:46561 every 8 connection(s)
I20260929 19:05:15.448079 28022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:15.448746 28100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:15.450695 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 28022
I20260929 19:05:15.450789 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:15.450951 26939 external_mini_cluster.cc:1428] Setting key 457316c6e387e5589e67556fdf18b79a
I20260929 19:05:15.451681 28100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4: Bootstrap starting.
I20260929 19:05:15.452273 28100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:15.452572 28100 log.cc:824] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:15.453262 28100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4: No bootstrap required, opened a new log
I20260929 19:05:15.455123 28100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdae13cdc20493e838da4c08233c9e4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46561 } }
I20260929 19:05:15.455288 28100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:15.455333 28100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fdae13cdc20493e838da4c08233c9e4, State: Initialized, Role: FOLLOWER
I20260929 19:05:15.455492 28100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdae13cdc20493e838da4c08233c9e4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46561 } }
I20260929 19:05:15.455585 28100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:15.455636 28100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:15.455682 28100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:15.456327 28100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdae13cdc20493e838da4c08233c9e4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46561 } }
I20260929 19:05:15.456413 28100 leader_election.cc:302] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fdae13cdc20493e838da4c08233c9e4; no voters:
I20260929 19:05:15.456586 28100 leader_election.cc:288] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:15.456612 28105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:15.456719 28105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [term 1 LEADER]: Becoming Leader. State: Replica: 9fdae13cdc20493e838da4c08233c9e4, State: Running, Role: LEADER
I20260929 19:05:15.456851 28100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:15.456854 28105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdae13cdc20493e838da4c08233c9e4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46561 } }
I20260929 19:05:15.457212 28105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fdae13cdc20493e838da4c08233c9e4. Latest consensus state: current_term: 1 leader_uuid: "9fdae13cdc20493e838da4c08233c9e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdae13cdc20493e838da4c08233c9e4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46561 } } }
I20260929 19:05:15.457222 28107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fdae13cdc20493e838da4c08233c9e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdae13cdc20493e838da4c08233c9e4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46561 } } }
I20260929 19:05:15.457396 28105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:15.457408 28107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4 [sys.catalog]: This master's current role is: LEADER
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:15.457675 28114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:15.458333 28114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:15.461942 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.451491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.26.78.254:46561)
0929 19:05:15.451666 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.451675 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.451707 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:05:15.452192 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.452199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.452248 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.452410 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.453113 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.454113 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.454870 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.455282 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.455413 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.458290 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.458294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.458295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.458307 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.460352 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.460736 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.460739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.460740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.460841 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.461057 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.461058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.461059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.461300 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.461467 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.461562 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.461697 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260929 19:05:15.463325 28114 catalog_manager.cc:1347] Generated new cluster ID: c6205e116c114e27bfcf561fb2dcec2e
I20260929 19:05:15.463380 28114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:15.478982 28114 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:15.479724 28114 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:15.491022 28114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4: Generated new TSK 0
I20260929 19:05:15.491258 28114 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:15.525368 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46561
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:15.608202 28126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:15.608425 28126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:15.608450 28126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:15.609629 28126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:15.609673 28126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:15.609695 28126 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:05:15.609757 28126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:15.611244 28126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:46561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:15.611532 28126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:15.611756 28126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:15.616703 28134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:15.616735 28132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:15.616834 28131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:15.617043 28126 server_base.cc:1034] running on GCE node
I20260929 19:05:15.617239 28126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:15.617470 28126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:15.618609 28126 hybrid_clock.cc:648] HybridClock initialized: now 1790708715618598 us; error 29 us; skew 500 ppm
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:15.621070 28126 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:15.621486 28126 webserver.cc:460] Webserver started at http://127.26.78.193:34425/ using document root <none> and password file <none>
I20260929 19:05:15.621726 28126 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:15.621794 28126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:15.621953 28126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:15.623030 28126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "9e663a09696c422e89edf6bd7925051e"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "c039dce4d55eceea5ba8b1e9da72744a"
server_key_iv: "e53784e91fe6c54bbb7871a74b303f75"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.623390 28126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "9e663a09696c422e89edf6bd7925051e"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "c039dce4d55eceea5ba8b1e9da72744a"
server_key_iv: "e53784e91fe6c54bbb7871a74b303f75"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.624662 28126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:15.625371 28141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.625540 28126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:15.625635 28126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "9e663a09696c422e89edf6bd7925051e"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "c039dce4d55eceea5ba8b1e9da72744a"
server_key_iv: "e53784e91fe6c54bbb7871a74b303f75"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.625748 28126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:15.634809 28126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:15.636704 28126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:15.636890 28126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:15.637392 28126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:15.637456 28126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.637511 28126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:15.637552 28126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:15.645025 28126 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:36059
I20260929 19:05:15.645452 28126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:05:15.645437 28256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:36059 every 8 connection(s)
I20260929 19:05:15.647676 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.637944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:35855 (local address 127.26.78.254:46561)
0929 19:05:15.638001 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.638004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.638201 (+ 197us) server_negotiation.cc:407] Connection header received
0929 19:05:15.638879 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.638881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.638895 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.638968 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.639884 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.640488 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.641667 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.641950 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.642038 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.644557 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.644559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.644560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.644569 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.646226 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.646663 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.646665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.646665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.646729 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.646990 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.646991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.646991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.647115 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.647187 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.647487 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.647583 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:05:15.647859 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.637926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46561 (local address 127.26.78.193:35855)
0929 19:05:15.638087 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.638120 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.638652 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.639098 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.639104 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.639491 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:15.639800 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.639807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.640582 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.640583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.641579 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.641581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.642064 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.642065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.642191 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.642744 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:15.642756 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:15.644454 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:15.646332 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.646338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.646340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.646584 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.646816 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.646816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.646817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.646920 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.647189 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:15.647196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:15.647350 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.647465 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.647632 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260929 19:05:15.648407 28257 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46561
I20260929 19:05:15.648509 28257 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:15.648686 28257 heartbeater.cc:511] Master 127.26.78.254:46561 requested a full tablet report, sending...
I20260929 19:05:15.649250 28054 ts_manager.cc:194] Registered new tserver with Master: 9e663a09696c422e89edf6bd7925051e (127.26.78.193:36059)
I20260929 19:05:15.650066 28054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:35855
I20260929 19:05:15.650513 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 28126
I20260929 19:05:15.650619 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:15.650807 26939 external_mini_cluster.cc:1428] Setting key ea13f6ceff74e4c071829bc3f0585e60
I20260929 19:05:15.656895 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.649561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:54041 (local address 127.26.78.254:46561)
0929 19:05:15.649610 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.649612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.649982 (+ 370us) server_negotiation.cc:407] Connection header received
0929 19:05:15.650034 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.650035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.650043 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.650085 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.651108 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.651690 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.652763 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.653070 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.653146 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.654321 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.654322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.654323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.654329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.655710 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.656119 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.656119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.656120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.656165 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.656375 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.656375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.656375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.656466 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.656515 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.656760 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.656833 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:05:15.657018 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.649484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46561 (local address 127.26.78.193:54041)
0929 19:05:15.649909 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.649921 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.649979 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.650101 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.650101 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.650316 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:15.650412 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.650414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.651796 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.651798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.652672 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.652675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.653172 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.653173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.653241 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.653725 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:15.653729 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:15.654228 (+ 499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:15.655796 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.655797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.655797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.656043 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.656229 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.656229 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.656230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.656311 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.656529 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:15.656530 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:15.656587 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.656731 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.656859 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:15.676877 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46561
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:15.757346 28266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:15.757546 28266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:15.757570 28266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:15.758838 28266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:15.758884 28266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:15.758905 28266 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:05:15.758924 28266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:15.760310 28266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:46561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:15.760509 28266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:15.760686 28266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:15.765576 28271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:15.765596 28272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:15.765578 28274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:15.765982 28266 server_base.cc:1034] running on GCE node
I20260929 19:05:15.766264 28266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:15.766517 28266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:15.767684 28266 hybrid_clock.cc:648] HybridClock initialized: now 1790708715767668 us; error 28 us; skew 500 ppm
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:15.770305 28266 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:15.770735 28266 webserver.cc:460] Webserver started at http://127.26.78.194:39075/ using document root <none> and password file <none>
I20260929 19:05:15.771008 28266 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:15.771081 28266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:15.771245 28266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:15.772313 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "945e3912876b48df9daebf56e23eb130"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "e73a96a121f3208918992951cacfcd3a"
server_key_iv: "33c75b155ca92cf28a2aab308a16ed4b"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.772691 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "945e3912876b48df9daebf56e23eb130"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "e73a96a121f3208918992951cacfcd3a"
server_key_iv: "33c75b155ca92cf28a2aab308a16ed4b"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.774178 28266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:05:15.774930 28281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.775094 28266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:15.775211 28266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "945e3912876b48df9daebf56e23eb130"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "e73a96a121f3208918992951cacfcd3a"
server_key_iv: "33c75b155ca92cf28a2aab308a16ed4b"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.775290 28266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:15.789464 28266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:15.791917 28266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:15.792089 28266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:15.792629 28266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:15.792696 28266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.792738 28266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:15.792768 28266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.799522 28266 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:41917
I20260929 19:05:15.799592 28396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:41917 every 8 connection(s)
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:15.799971 28266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:15.801775 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 28266
I20260929 19:05:15.801898 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:15.802085 26939 external_mini_cluster.cc:1428] Setting key cd10bc8b0bd90aa332b3037be0e5e710
I20260929 19:05:15.803323 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.793145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:47811 (local address 127.26.78.254:46561)
0929 19:05:15.793229 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.793232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.793434 (+ 202us) server_negotiation.cc:407] Connection header received
0929 19:05:15.793972 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.793974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.793988 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.794065 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.794885 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.795741 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.797379 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.797724 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.797806 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.800480 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.800482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.800482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.800489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.801660 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.802197 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.802198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.802199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.802266 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.802576 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.802577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.802578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.802738 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.802808 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.803178 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.803243 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 19:05:15.803633 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.793116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46561 (local address 127.26.78.194:47811)
0929 19:05:15.793266 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.793308 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.793809 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.794137 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.794141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.794459 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:15.794765 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.794771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.795873 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.795874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.797272 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.797275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.797859 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.797860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.798022 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.798595 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:15.798607 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:15.800400 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:15.801743 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.801751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.801753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.802082 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.802363 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.802364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.802364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.802501 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.802813 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:15.802819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:15.803024 (+ 205us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.803137 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.803353 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260929 19:05:15.804129 28397 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46561
I20260929 19:05:15.804286 28397 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:15.804522 28397 heartbeater.cc:511] Master 127.26.78.254:46561 requested a full tablet report, sending...
I20260929 19:05:15.804885 28053 ts_manager.cc:194] Registered new tserver with Master: 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917)
I20260929 19:05:15.805438 28053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:47811
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
I20260929 19:05:15.810946 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.805247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:55559 (local address 127.26.78.254:46561)
0929 19:05:15.805293 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.805296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.805344 (+ 48us) server_negotiation.cc:407] Connection header received
0929 19:05:15.805399 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.805400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.805409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.805471 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.805899 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.806380 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.807173 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.807383 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.807465 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.808411 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.808412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.808412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.808419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.809695 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.810068 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.810069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.810070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.810128 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.810341 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.810342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.810342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.810550 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.810613 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.810775 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.810858 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:15.810961 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.805169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46561 (local address 127.26.78.194:55559)
0929 19:05:15.805293 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.805302 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.805353 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.805493 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.805494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.805717 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:15.805785 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.805787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.806477 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.806478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.807076 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.807078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.807468 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.807469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.807541 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.808029 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:15.808034 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:15.808323 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:15.809788 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.809788 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.809789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.809992 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.810205 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.810206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.810206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.810271 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.810616 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:15.810617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:15.810662 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.810749 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.810853 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:15.828081 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46561
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:15.909077 28406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:15.909313 28406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:15.909354 28406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:15.910573 28406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:15.910636 28406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:15.910673 28406 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:05:15.910714 28406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:15.912141 28406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:46561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:15.912403 28406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:15.912631 28406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:15.917563 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:15.917554 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:15.917536 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:15.917809 28406 server_base.cc:1034] running on GCE node
I20260929 19:05:15.918066 28406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:15.918325 28406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:15.919473 28406 hybrid_clock.cc:648] HybridClock initialized: now 1790708715919458 us; error 43 us; skew 500 ppm
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:15.921957 28406 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:15.922411 28406 webserver.cc:460] Webserver started at http://127.26.78.195:45209/ using document root <none> and password file <none>
I20260929 19:05:15.922658 28406 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:15.922703 28406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:15.922851 28406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:15.923894 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "a3c29510c5564be092d7acc6d9a2e762"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "a363dcb5a4c80cb88d46438e5ee68366"
server_key_iv: "d02322c91f9104bfb36468048d735f69"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.924247 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "a3c29510c5564be092d7acc6d9a2e762"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "a363dcb5a4c80cb88d46438e5ee68366"
server_key_iv: "d02322c91f9104bfb36468048d735f69"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.925524 28406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:15.926340 28421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.926512 28406 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.926609 28406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "a3c29510c5564be092d7acc6d9a2e762"
format_stamp: "Formatted at 2026-09-29 19:05:15 on dist-test-slave-xz51"
server_key: "a363dcb5a4c80cb88d46438e5ee68366"
server_key_iv: "d02322c91f9104bfb36468048d735f69"
server_key_version: "encryptionkey@0"
I20260929 19:05:15.926690 28406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:15.940018 28406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:15.941962 28406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:15.942126 28406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:15.942647 28406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:15.942714 28406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.942768 28406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:15.942791 28406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:15.949970 28406 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:44119
I20260929 19:05:15.950016 28536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:44119 every 8 connection(s)
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:15.950380 28406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:05:15.953097 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 28406
I20260929 19:05:15.953204 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:15.953420 26939 external_mini_cluster.cc:1428] Setting key 8949f69f8ee22692a76c69a474cca94c
I20260929 19:05:15.954016 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.943211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46561 (local address 127.26.78.195:59869)
0929 19:05:15.943392 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.943427 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.943958 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.944209 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.944213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.944629 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:15.944929 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.944936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.946036 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.946037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.946877 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.946879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.947408 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.947410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.947587 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.948833 (+ 1246us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:15.948845 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:15.950769 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:15.952301 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.952307 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.952309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.952522 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.952769 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.952770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.952770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.952866 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.953235 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:15.953242 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:15.953417 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.953545 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.953714 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":117,"threads_started":1}
I20260929 19:05:15.954353 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.943217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:59869 (local address 127.26.78.254:46561)
0929 19:05:15.943259 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.943261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.943516 (+ 255us) server_negotiation.cc:407] Connection header received
0929 19:05:15.944071 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.944072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.944085 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.944132 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.945045 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.945931 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.946974 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.947300 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.947373 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.950874 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.950876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.950876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.950885 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.952201 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.952642 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.952643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.952643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.952705 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.952943 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.952944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.952944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.953116 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.953199 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.954176 (+ 977us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.954268 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:05:15.954815 28537 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46561
I20260929 19:05:15.954903 28537 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:15.955106 28537 heartbeater.cc:511] Master 127.26.78.254:46561 requested a full tablet report, sending...
I20260929 19:05:15.955504 28054 ts_manager.cc:194] Registered new tserver with Master: a3c29510c5564be092d7acc6d9a2e762 (127.26.78.195:44119)
I20260929 19:05:15.955976 28054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:59869
I20260929 19:05:15.961664 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.955910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:41249 (local address 127.26.78.254:46561)
0929 19:05:15.955952 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.955954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.956070 (+ 116us) server_negotiation.cc:407] Connection header received
0929 19:05:15.956267 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.956268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.956293 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.956380 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.956773 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.957340 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.958201 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.958426 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.958504 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.959427 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.959429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.959429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.959435 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.960636 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.960972 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.960973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.960973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.961021 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.961210 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.961211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.961211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.961321 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.961373 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.961543 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.961602 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:05:15.961740 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.955847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46561 (local address 127.26.78.195:41249)
0929 19:05:15.955994 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.956004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.956066 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.956356 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.956358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.956636 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:15.956691 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.956693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.957428 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.957429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.958115 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.958118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.958542 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.958543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.958653 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.959062 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:15.959066 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:15.959351 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:15.960724 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.960725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.960726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.960901 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.961103 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:15.961104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:15.961104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:15.961155 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:15.961402 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:15.961404 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:15.961447 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.961517 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.961602 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260929 19:05:15.965410 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:05:15 dist-test-slave-xz51 krb5kdc[28009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:15.988302 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.981097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.26.78.254:46561)
0929 19:05:15.981171 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.981173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.981183 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:15.981216 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.981217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.981226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.981284 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:15.981885 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.982388 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.983146 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.983338 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.983415 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.985929 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:15.985930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:15.985930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:15.985938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:15.987095 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.987475 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.987476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.987476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.987534 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:15.987735 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:15.987736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:15.987737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:15.987851 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:15.987955 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.988094 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.988193 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:05:15.989871 28054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:15.991544 28054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:15.998148 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.993959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36059 (local address 127.0.0.1:56106)
0929 19:05:15.994242 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.994253 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.994338 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.994589 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.994590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.994592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:15.994671 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.994673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.995680 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.995681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.996913 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.996916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.997769 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.997770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.997888 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.997902 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.998003 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.998060 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260929 19:05:15.998196 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.993982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.26.78.193:36059)
0929 19:05:15.994155 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.994160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.994308 (+ 148us) server_negotiation.cc:407] Connection header received
0929 19:05:15.994432 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.994434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.994448 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.994519 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:15.994755 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.995596 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.996998 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.997665 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.997767 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.997849 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.998047 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.998087 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260929 19:05:15.998268 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.993897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:44119 (local address 127.0.0.1:51820)
0929 19:05:15.994024 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.994072 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.994143 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.994564 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.994568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.994570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:15.994706 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.994709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.996074 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.996075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.997122 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.997123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.998056 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.998057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.998121 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.998124 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:15.998154 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 19:05:15.998197 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":34,"thread_start_us":66,"threads_started":1}
I20260929 19:05:15.998653 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.994173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51820 (local address 127.26.78.195:44119)
0929 19:05:15.994363 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.994368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.994378 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:15.994417 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.994420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.994436 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.994549 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:15.994799 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.995973 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.997206 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.997966 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.998414 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.998484 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:15.998512 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 19:05:15.998558 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260929 19:05:15.999146 28191 tablet_service.cc:1467] Processing CreateTablet for tablet 68935ea567b64382b090e7cb748bae05 (DEFAULT_TABLE table=test-table [id=1bda54a2243d44139a81340ef45bb065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:15.999272 28471 tablet_service.cc:1467] Processing CreateTablet for tablet 68935ea567b64382b090e7cb748bae05 (DEFAULT_TABLE table=test-table [id=1bda54a2243d44139a81340ef45bb065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:15.999583 28191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68935ea567b64382b090e7cb748bae05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:15.999639 28471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68935ea567b64382b090e7cb748bae05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.002099 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.994138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41917 (local address 127.0.0.1:43708)
0929 19:05:15.994513 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:15.994522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:15.994568 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:15.994935 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:15.994936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:15.994938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:15.994970 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.994971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.996190 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.996191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.997253 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:15.997255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.998196 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.998197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:15.998710 (+ 513us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.001876 (+ 3166us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.001942 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.001995 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260929 19:05:16.002363 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:15.994351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.26.78.194:41917)
0929 19:05:15.994495 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:15.994499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:15.994662 (+ 163us) server_negotiation.cc:407] Connection header received
0929 19:05:15.994759 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:15.994761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:15.994778 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:15.994856 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:15.995042 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.996093 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.997335 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:15.998052 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:15.998111 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:15.998166 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.002207 (+ 4041us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.002260 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260929 19:05:16.002650 28561 tablet_bootstrap.cc:492] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Bootstrap starting.
I20260929 19:05:16.002697 28560 tablet_bootstrap.cc:492] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Bootstrap starting.
I20260929 19:05:16.003034 28331 tablet_service.cc:1467] Processing CreateTablet for tablet 68935ea567b64382b090e7cb748bae05 (DEFAULT_TABLE table=test-table [id=1bda54a2243d44139a81340ef45bb065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:16.003350 28561 tablet_bootstrap.cc:654] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.003371 28331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68935ea567b64382b090e7cb748bae05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.003504 28560 tablet_bootstrap.cc:654] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.003681 28561 log.cc:824] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:16.003873 28560 log.cc:824] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:16.004415 28561 tablet_bootstrap.cc:492] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: No bootstrap required, opened a new log
I20260929 19:05:16.004487 28561 ts_tablet_manager.cc:1397] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:16.004704 28560 tablet_bootstrap.cc:492] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: No bootstrap required, opened a new log
I20260929 19:05:16.004772 28560 ts_tablet_manager.cc:1397] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:05:16.006588 28561 raft_consensus.cc:348] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.006778 28561 raft_consensus.cc:374] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.006816 28561 raft_consensus.cc:729] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e663a09696c422e89edf6bd7925051e, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.006754 28564 tablet_bootstrap.cc:492] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Bootstrap starting.
I20260929 19:05:16.006915 28561 consensus_queue.cc:260] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.006996 28560 raft_consensus.cc:348] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.007169 28560 raft_consensus.cc:374] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.007186 28561 ts_tablet_manager.cc:1428] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:05:16.007215 28560 raft_consensus.cc:729] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c29510c5564be092d7acc6d9a2e762, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.007215 28257 heartbeater.cc:503] Master 127.26.78.254:46561 was elected leader, sending a full tablet report...
I20260929 19:05:16.007325 28560 consensus_queue.cc:260] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.007591 28560 ts_tablet_manager.cc:1428] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:05:16.007604 28537 heartbeater.cc:503] Master 127.26.78.254:46561 was elected leader, sending a full tablet report...
I20260929 19:05:16.007827 28564 tablet_bootstrap.cc:654] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.008174 28564 log.cc:824] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:16.009001 28564 tablet_bootstrap.cc:492] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: No bootstrap required, opened a new log
I20260929 19:05:16.009056 28564 ts_tablet_manager.cc:1397] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:16.010636 28564 raft_consensus.cc:348] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.010758 28564 raft_consensus.cc:374] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.010779 28564 raft_consensus.cc:729] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945e3912876b48df9daebf56e23eb130, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.010892 28564 consensus_queue.cc:260] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.011116 28564 ts_tablet_manager.cc:1428] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:05:16.011201 28397 heartbeater.cc:503] Master 127.26.78.254:46561 was elected leader, sending a full tablet report...
W20260929 19:05:16.050634 28398 tablet.cc:2369] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:05:16.094959 28566 raft_consensus.cc:482] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:16.095098 28566 raft_consensus.cc:504] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.095459 28566 leader_election.cc:288] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e663a09696c422e89edf6bd7925051e (127.26.78.193:36059), 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917)
I20260929 19:05:16.098332 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.095648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36059 (local address 127.26.78.195:41739)
0929 19:05:16.095716 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.095729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.095797 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.095965 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.095967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.095969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:16.096047 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.096049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.096701 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.096702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.097441 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.097443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.098056 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.098057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.098149 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.098152 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.098217 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.098255 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 19:05:16.098325 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.095745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:41739 (local address 127.26.78.193:36059)
0929 19:05:16.095827 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.095830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.095842 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:05:16.095873 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.095874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.095886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.095956 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:16.096128 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.096610 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.097538 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.097976 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.098060 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.098070 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.098222 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.098260 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:05:16.098449 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.095775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41917 (local address 127.26.78.195:40309)
0929 19:05:16.095932 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.095943 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.095995 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.096170 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.096170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.096172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:16.096251 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.096252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.096899 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.096900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.097605 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.097607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.098180 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.098181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.098259 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.098262 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.098331 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.098370 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
I20260929 19:05:16.098443 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.095907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:40309 (local address 127.26.78.194:41917)
0929 19:05:16.095956 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.095958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.095992 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:05:16.096075 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.096076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.096094 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.096165 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:16.096346 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.096807 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.097704 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.098090 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.098179 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.098186 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.098338 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.098377 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:16.098829 28351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68935ea567b64382b090e7cb748bae05" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945e3912876b48df9daebf56e23eb130" is_pre_election: true
I20260929 19:05:16.098848 28211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68935ea567b64382b090e7cb748bae05" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e663a09696c422e89edf6bd7925051e" is_pre_election: true
I20260929 19:05:16.098989 28211 raft_consensus.cc:2454] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 0.
I20260929 19:05:16.098989 28351 raft_consensus.cc:2454] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 0.
I20260929 19:05:16.099224 28424 leader_election.cc:302] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e663a09696c422e89edf6bd7925051e, a3c29510c5564be092d7acc6d9a2e762; no voters:
I20260929 19:05:16.099360 28566 raft_consensus.cc:2781] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:16.099443 28566 raft_consensus.cc:482] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:16.099478 28566 raft_consensus.cc:3037] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.100199 28566 raft_consensus.cc:504] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.100363 28566 leader_election.cc:288] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 election: Requested vote from peers 9e663a09696c422e89edf6bd7925051e (127.26.78.193:36059), 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917)
I20260929 19:05:16.100531 28211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68935ea567b64382b090e7cb748bae05" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e663a09696c422e89edf6bd7925051e"
I20260929 19:05:16.100576 28351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68935ea567b64382b090e7cb748bae05" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945e3912876b48df9daebf56e23eb130"
I20260929 19:05:16.100620 28211 raft_consensus.cc:3037] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.100648 28351 raft_consensus.cc:3037] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.101519 28351 raft_consensus.cc:2454] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 1.
I20260929 19:05:16.101519 28211 raft_consensus.cc:2454] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 1.
I20260929 19:05:16.101745 28424 leader_election.cc:302] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 945e3912876b48df9daebf56e23eb130, a3c29510c5564be092d7acc6d9a2e762; no voters:
I20260929 19:05:16.101861 28566 raft_consensus.cc:2781] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:16.102015 28566 raft_consensus.cc:686] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 1 LEADER]: Becoming Leader. State: Replica: a3c29510c5564be092d7acc6d9a2e762, State: Running, Role: LEADER
I20260929 19:05:16.102113 28566 consensus_queue.cc:237] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } }
I20260929 19:05:16.102995 28054 catalog_manager.cc:5495] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3c29510c5564be092d7acc6d9a2e762 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "a3c29510c5564be092d7acc6d9a2e762" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } health_report { overall_health: UNKNOWN } } }
I20260929 19:05:16.114892 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.112027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51834 (local address 127.26.78.195:44119)
0929 19:05:16.112094 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.112096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.112105 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:16.112171 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.112171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.112190 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.112294 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:16.112461 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.112974 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.113773 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.114034 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.114113 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.114328 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:16.114451 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:16.114537 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.114590 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.114613 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:16.116667 28351 raft_consensus.cc:1261] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Refusing update from remote peer a3c29510c5564be092d7acc6d9a2e762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:16.116683 28211 raft_consensus.cc:1261] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Refusing update from remote peer a3c29510c5564be092d7acc6d9a2e762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:16.117051 28566 consensus_queue.cc:1035] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:16.117249 28570 consensus_queue.cc:1035] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:16.119568 28574 mvcc.cc:205] Tried to move back new op lower bound from 7334742901211209728 to 7334742901155119104. Current Snapshot: MvccSnapshot[applied={T|T < 7334742901211209728}]
I20260929 19:05:16.123713 28053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:16.123871 28053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:16.124671 28053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4: Sending DeleteTablet for 3 replicas of tablet 68935ea567b64382b090e7cb748bae05
I20260929 19:05:16.125026 28471 tablet_service.cc:1514] Processing DeleteTablet for tablet 68935ea567b64382b090e7cb748bae05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:16 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:51820
I20260929 19:05:16.125139 28331 tablet_service.cc:1514] Processing DeleteTablet for tablet 68935ea567b64382b090e7cb748bae05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:16 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:43708
I20260929 19:05:16.125252 28191 tablet_service.cc:1514] Processing DeleteTablet for tablet 68935ea567b64382b090e7cb748bae05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:16 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:56106
I20260929 19:05:16.125453 28585 tablet_replica.cc:330] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: stopping tablet replica
I20260929 19:05:16.125615 28585 raft_consensus.cc:2229] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:16.125752 28585 raft_consensus.cc:2258] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:16.125679 28053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:16.125869 28053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:16.126300 28585 ts_tablet_manager.cc:1905] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:16.126367 28583 tablet_replica.cc:330] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: stopping tablet replica
I20260929 19:05:16.126448 28583 raft_consensus.cc:2229] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:16.126586 28583 raft_consensus.cc:2258] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:16.127378 28584 tablet_replica.cc:330] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: stopping tablet replica
I20260929 19:05:16.127486 28584 raft_consensus.cc:2229] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:16.127640 28331 tablet_service.cc:1467] Processing CreateTablet for tablet 0b763519e8a54cfc92721d4cc9d5c19e (DEFAULT_TABLE table=test-table [id=ed199597ba594b879ed2ba841c146121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:16.127761 28331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b763519e8a54cfc92721d4cc9d5c19e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.127841 28584 raft_consensus.cc:2258] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:16.128154 28583 ts_tablet_manager.cc:1905] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:16.128921 28585 ts_tablet_manager.cc:1918] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:16.128983 28585 log.cc:1196] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/68935ea567b64382b090e7cb748bae05
I20260929 19:05:16.129137 28564 tablet_bootstrap.cc:492] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: Bootstrap starting.
I20260929 19:05:16.129590 28564 tablet_bootstrap.cc:654] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.129578 28191 tablet_service.cc:1467] Processing CreateTablet for tablet 0b763519e8a54cfc92721d4cc9d5c19e (DEFAULT_TABLE table=test-table [id=ed199597ba594b879ed2ba841c146121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:16.129683 28191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b763519e8a54cfc92721d4cc9d5c19e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.129921 28583 ts_tablet_manager.cc:1918] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:16.129987 28583 log.cc:1196] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/68935ea567b64382b090e7cb748bae05
I20260929 19:05:16.130162 28585 ts_tablet_manager.cc:1939] T 68935ea567b64382b090e7cb748bae05 P 9e663a09696c422e89edf6bd7925051e: Deleting consensus metadata
I20260929 19:05:16.130223 28583 ts_tablet_manager.cc:1939] T 68935ea567b64382b090e7cb748bae05 P a3c29510c5564be092d7acc6d9a2e762: Deleting consensus metadata
I20260929 19:05:16.130563 28040 catalog_manager.cc:4841] TS 9e663a09696c422e89edf6bd7925051e (127.26.78.193:36059): tablet 68935ea567b64382b090e7cb748bae05 (table test-table [id=1bda54a2243d44139a81340ef45bb065]) successfully deleted
I20260929 19:05:16.130878 28584 ts_tablet_manager.cc:1905] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:16.130966 28564 tablet_bootstrap.cc:492] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: No bootstrap required, opened a new log
I20260929 19:05:16.131004 28564 ts_tablet_manager.cc:1397] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:16.131016 28561 tablet_bootstrap.cc:492] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Bootstrap starting.
I20260929 19:05:16.131153 28564 raft_consensus.cc:348] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.131235 28564 raft_consensus.cc:374] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.131256 28564 raft_consensus.cc:729] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945e3912876b48df9daebf56e23eb130, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.131296 28564 consensus_queue.cc:260] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.131388 28564 ts_tablet_manager.cc:1428] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:16.131446 28561 tablet_bootstrap.cc:654] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.131937 28039 catalog_manager.cc:4841] TS a3c29510c5564be092d7acc6d9a2e762 (127.26.78.195:44119): tablet 68935ea567b64382b090e7cb748bae05 (table test-table [id=1bda54a2243d44139a81340ef45bb065]) successfully deleted
I20260929 19:05:16.132030 28561 tablet_bootstrap.cc:492] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: No bootstrap required, opened a new log
I20260929 19:05:16.132067 28561 ts_tablet_manager.cc:1397] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:16.132205 28561 raft_consensus.cc:348] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.132264 28561 raft_consensus.cc:374] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.132293 28561 raft_consensus.cc:729] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e663a09696c422e89edf6bd7925051e, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.132258 28471 tablet_service.cc:1467] Processing CreateTablet for tablet 0b763519e8a54cfc92721d4cc9d5c19e (DEFAULT_TABLE table=test-table [id=ed199597ba594b879ed2ba841c146121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:16.132375 28471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b763519e8a54cfc92721d4cc9d5c19e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.132347 28561 consensus_queue.cc:260] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.132414 28584 ts_tablet_manager.cc:1918] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:16.132457 28561 ts_tablet_manager.cc:1428] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:16.132472 28584 log.cc:1196] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/68935ea567b64382b090e7cb748bae05
I20260929 19:05:16.132740 28584 ts_tablet_manager.cc:1939] T 68935ea567b64382b090e7cb748bae05 P 945e3912876b48df9daebf56e23eb130: Deleting consensus metadata
I20260929 19:05:16.133186 28040 catalog_manager.cc:4841] TS 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917): tablet 68935ea567b64382b090e7cb748bae05 (table test-table [id=1bda54a2243d44139a81340ef45bb065]) successfully deleted
I20260929 19:05:16.133577 28560 tablet_bootstrap.cc:492] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Bootstrap starting.
I20260929 19:05:16.133940 28560 tablet_bootstrap.cc:654] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.134578 28560 tablet_bootstrap.cc:492] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: No bootstrap required, opened a new log
I20260929 19:05:16.134626 28560 ts_tablet_manager.cc:1397] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:16.134756 28560 raft_consensus.cc:348] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.134809 28560 raft_consensus.cc:374] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.134822 28560 raft_consensus.cc:729] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c29510c5564be092d7acc6d9a2e762, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.134882 28560 consensus_queue.cc:260] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.134984 28560 ts_tablet_manager.cc:1428] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260929 19:05:16.146236 28258 tablet.cc:2369] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:05:16.201130 28538 tablet.cc:2369] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:05:16.430685 28570 raft_consensus.cc:482] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:16.430794 28570 raft_consensus.cc:504] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.430975 28570 leader_election.cc:288] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e663a09696c422e89edf6bd7925051e (127.26.78.193:36059), 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917)
I20260929 19:05:16.431211 28211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b763519e8a54cfc92721d4cc9d5c19e" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e663a09696c422e89edf6bd7925051e" is_pre_election: true
I20260929 19:05:16.431331 28211 raft_consensus.cc:2454] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 0.
I20260929 19:05:16.431288 28351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b763519e8a54cfc92721d4cc9d5c19e" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945e3912876b48df9daebf56e23eb130" is_pre_election: true
I20260929 19:05:16.431427 28351 raft_consensus.cc:2454] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 0.
I20260929 19:05:16.431520 28424 leader_election.cc:302] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e663a09696c422e89edf6bd7925051e, a3c29510c5564be092d7acc6d9a2e762; no voters:
I20260929 19:05:16.431609 28570 raft_consensus.cc:2781] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:16.431653 28570 raft_consensus.cc:482] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:16.431672 28570 raft_consensus.cc:3037] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.432379 28570 raft_consensus.cc:504] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.432513 28570 leader_election.cc:288] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 election: Requested vote from peers 9e663a09696c422e89edf6bd7925051e (127.26.78.193:36059), 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917)
I20260929 19:05:16.432690 28351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b763519e8a54cfc92721d4cc9d5c19e" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945e3912876b48df9daebf56e23eb130"
I20260929 19:05:16.432763 28351 raft_consensus.cc:3037] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.432719 28211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b763519e8a54cfc92721d4cc9d5c19e" candidate_uuid: "a3c29510c5564be092d7acc6d9a2e762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e663a09696c422e89edf6bd7925051e"
I20260929 19:05:16.432818 28211 raft_consensus.cc:3037] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.433432 28211 raft_consensus.cc:2454] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 1.
I20260929 19:05:16.433434 28351 raft_consensus.cc:2454] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c29510c5564be092d7acc6d9a2e762 in term 1.
I20260929 19:05:16.433617 28424 leader_election.cc:302] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e663a09696c422e89edf6bd7925051e, a3c29510c5564be092d7acc6d9a2e762; no voters:
I20260929 19:05:16.433717 28570 raft_consensus.cc:2781] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:16.433784 28570 raft_consensus.cc:686] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 1 LEADER]: Becoming Leader. State: Replica: a3c29510c5564be092d7acc6d9a2e762, State: Running, Role: LEADER
I20260929 19:05:16.433862 28570 consensus_queue.cc:237] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.434423 28054 catalog_manager.cc:5495] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3c29510c5564be092d7acc6d9a2e762 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "a3c29510c5564be092d7acc6d9a2e762" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } health_report { overall_health: HEALTHY } } }
Sep 29 19:05:16 dist-test-slave-xz51 krb5kdc[28009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:16.505492 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.498573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.26.78.254:46561)
0929 19:05:16.498686 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.498688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.498772 (+ 84us) server_negotiation.cc:407] Connection header received
0929 19:05:16.498813 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.498814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.498824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.498885 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:16.499311 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.499812 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.500520 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.500703 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.500753 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.502830 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:16.502832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:16.502832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:16.502838 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:16.504443 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:16.504829 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:16.504830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:16.504831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:16.504884 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:16.505066 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:16.505067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:16.505067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:16.505179 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:16.505230 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.505346 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.505419 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260929 19:05:16.505707 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.498515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46561 (local address 127.0.0.1:41820)
0929 19:05:16.498693 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.498703 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.498766 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.498901 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.498902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.499191 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:16.499237 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.499238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.499915 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.499917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.500450 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.500451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.500787 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.500789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.500835 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.501329 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:16.501333 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:16.502754 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:16.504556 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:16.504570 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:16.504572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:16.504761 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:16.504942 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:16.504943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:16.504943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:16.504999 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:16.505233 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:16.505239 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:16.505277 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.505330 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.505488 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":68,"thread_start_us":92,"threads_started":1}
I20260929 19:05:16.507613 28054 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41820:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 19:05:16.507822 28054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:16.509298 28331 tablet_service.cc:1467] Processing CreateTablet for tablet dfd999e4ae754b949c92f1f124d0e11e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6bb0c514092c4c85b2b8a9498089e97a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:05:16.509302 28191 tablet_service.cc:1467] Processing CreateTablet for tablet dfd999e4ae754b949c92f1f124d0e11e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6bb0c514092c4c85b2b8a9498089e97a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:05:16.509299 28471 tablet_service.cc:1467] Processing CreateTablet for tablet dfd999e4ae754b949c92f1f124d0e11e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6bb0c514092c4c85b2b8a9498089e97a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:05:16.509454 28331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd999e4ae754b949c92f1f124d0e11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.509457 28471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd999e4ae754b949c92f1f124d0e11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.509459 28191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd999e4ae754b949c92f1f124d0e11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:16.510584 28564 tablet_bootstrap.cc:492] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: Bootstrap starting.
I20260929 19:05:16.510941 28561 tablet_bootstrap.cc:492] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: Bootstrap starting.
I20260929 19:05:16.510946 28560 tablet_bootstrap.cc:492] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: Bootstrap starting.
I20260929 19:05:16.511003 28564 tablet_bootstrap.cc:654] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.511307 28560 tablet_bootstrap.cc:654] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.511376 28561 tablet_bootstrap.cc:654] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:16.511760 28564 tablet_bootstrap.cc:492] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: No bootstrap required, opened a new log
I20260929 19:05:16.511829 28564 ts_tablet_manager.cc:1397] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:16.511957 28561 tablet_bootstrap.cc:492] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: No bootstrap required, opened a new log
I20260929 19:05:16.512006 28561 ts_tablet_manager.cc:1397] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:16.511989 28564 raft_consensus.cc:348] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.512085 28564 raft_consensus.cc:374] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.512116 28564 raft_consensus.cc:729] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945e3912876b48df9daebf56e23eb130, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.512167 28564 consensus_queue.cc:260] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.512161 28561 raft_consensus.cc:348] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.512221 28561 raft_consensus.cc:374] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.512251 28561 raft_consensus.cc:729] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e663a09696c422e89edf6bd7925051e, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.512264 28564 ts_tablet_manager.cc:1428] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:16.512306 28561 consensus_queue.cc:260] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.512312 28568 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.512403 28561 ts_tablet_manager.cc:1428] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:16.512403 28568 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:16.512526 28565 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.512603 28565 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:16.512655 28560 tablet_bootstrap.cc:492] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: No bootstrap required, opened a new log
I20260929 19:05:16.512698 28560 ts_tablet_manager.cc:1397] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:05:16.512809 28560 raft_consensus.cc:348] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.512879 28560 raft_consensus.cc:374] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:16.512898 28560 raft_consensus.cc:729] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c29510c5564be092d7acc6d9a2e762, State: Initialized, Role: FOLLOWER
I20260929 19:05:16.512943 28560 consensus_queue.cc:260] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.513034 28560 ts_tablet_manager.cc:1428] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:16.513052 28570 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.513155 28570 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:16.533763 28565 raft_consensus.cc:482] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:16.533946 28565 raft_consensus.cc:504] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.534236 28565 leader_election.cc:288] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917), a3c29510c5564be092d7acc6d9a2e762 (127.26.78.195:44119)
I20260929 19:05:16.538422 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.534521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:38613 (local address 127.26.78.194:41917)
0929 19:05:16.534574 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.534576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.534656 (+ 80us) server_negotiation.cc:407] Connection header received
0929 19:05:16.534698 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.534699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.534709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.534799 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:16.534999 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.535739 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.537405 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.537994 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.538056 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.538065 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.538295 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.538347 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:05:16.538457 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.534424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:44119 (local address 127.26.78.193:40385)
0929 19:05:16.534580 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.534588 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.534647 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.534808 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.534809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.534810 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:16.534917 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.534920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.535839 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.535840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.537298 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.537302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.538147 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.538149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.538243 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.538248 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.538312 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.538351 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":102,"spinlock_wait_cycles":896,"thread_start_us":64,"threads_started":1}
I20260929 19:05:16.538467 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.534502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:40385 (local address 127.26.78.195:44119)
0929 19:05:16.534551 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.534553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.534656 (+ 103us) server_negotiation.cc:407] Connection header received
0929 19:05:16.534698 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.534699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.534711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.534776 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:16.534999 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.535742 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.537405 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.537983 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.538058 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.538065 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.538349 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.538403 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:16.538463 28601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.534424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41917 (local address 127.26.78.193:38613)
0929 19:05:16.534565 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.534577 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.534647 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.534802 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.534803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.534806 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:16.534917 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.534919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.535840 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.535840 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.537312 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.537314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.538138 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.538139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.538218 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.538221 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.538266 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.538324 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":99,"thread_start_us":79,"threads_started":1}
I20260929 19:05:16.538928 28351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd999e4ae754b949c92f1f124d0e11e" candidate_uuid: "9e663a09696c422e89edf6bd7925051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945e3912876b48df9daebf56e23eb130" is_pre_election: true
I20260929 19:05:16.539021 28351 raft_consensus.cc:2454] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e663a09696c422e89edf6bd7925051e in term 0.
I20260929 19:05:16.538944 28491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd999e4ae754b949c92f1f124d0e11e" candidate_uuid: "9e663a09696c422e89edf6bd7925051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c29510c5564be092d7acc6d9a2e762" is_pre_election: true
I20260929 19:05:16.539094 28491 raft_consensus.cc:2454] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e663a09696c422e89edf6bd7925051e in term 0.
I20260929 19:05:16.539193 28144 leader_election.cc:302] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 945e3912876b48df9daebf56e23eb130, 9e663a09696c422e89edf6bd7925051e; no voters:
I20260929 19:05:16.539354 28565 raft_consensus.cc:2781] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:16.539408 28565 raft_consensus.cc:482] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:16.539424 28565 raft_consensus.cc:3037] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.540087 28565 raft_consensus.cc:504] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.540230 28565 leader_election.cc:288] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [CANDIDATE]: Term 1 election: Requested vote from peers 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917), a3c29510c5564be092d7acc6d9a2e762 (127.26.78.195:44119)
I20260929 19:05:16.540372 28351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd999e4ae754b949c92f1f124d0e11e" candidate_uuid: "9e663a09696c422e89edf6bd7925051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945e3912876b48df9daebf56e23eb130"
I20260929 19:05:16.540391 28491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd999e4ae754b949c92f1f124d0e11e" candidate_uuid: "9e663a09696c422e89edf6bd7925051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c29510c5564be092d7acc6d9a2e762"
I20260929 19:05:16.540445 28351 raft_consensus.cc:3037] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.540452 28491 raft_consensus.cc:3037] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:16.541143 28351 raft_consensus.cc:2454] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e663a09696c422e89edf6bd7925051e in term 1.
I20260929 19:05:16.541263 28491 raft_consensus.cc:2454] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e663a09696c422e89edf6bd7925051e in term 1.
I20260929 19:05:16.541320 28144 leader_election.cc:302] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 945e3912876b48df9daebf56e23eb130, 9e663a09696c422e89edf6bd7925051e; no voters:
I20260929 19:05:16.541405 28565 raft_consensus.cc:2781] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:16.541507 28565 raft_consensus.cc:686] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [term 1 LEADER]: Becoming Leader. State: Replica: 9e663a09696c422e89edf6bd7925051e, State: Running, Role: LEADER
I20260929 19:05:16.541594 28565 consensus_queue.cc:237] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } }
I20260929 19:05:16.542021 28602 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: TxnStatusTablet state changed. Reason: New leader 9e663a09696c422e89edf6bd7925051e. Latest consensus state: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.542122 28602 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:05:16.542308 28604 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 19:05:16.542400 28604 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 19:05:16.542351 28054 catalog_manager.cc:5495] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e663a09696c422e89edf6bd7925051e (127.26.78.193). New cstate: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } health_report { overall_health: UNKNOWN } } }
I20260929 19:05:16.550531 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.547920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56114 (local address 127.26.78.193:36059)
0929 19:05:16.547978 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.547980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.548018 (+ 38us) server_negotiation.cc:407] Connection header received
0929 19:05:16.548118 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.548118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.548138 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.548203 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:16.548329 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.548807 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.549642 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.549915 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.550004 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.550194 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:16.550307 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:16.550384 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.550440 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.550462 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:05:16.550556 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.547857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36059 (local address 127.0.0.1:56114)
0929 19:05:16.547956 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.547968 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.548039 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.548211 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.548212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.548213 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:16.548263 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.548265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.548901 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.548902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.549558 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.549561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.550005 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.550006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.550074 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.550089 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:16.550383 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:16.550393 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.550438 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.550484 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 19:05:16.551643 28351 raft_consensus.cc:1261] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Refusing update from remote peer 9e663a09696c422e89edf6bd7925051e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:16.551710 28491 raft_consensus.cc:1261] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Refusing update from remote peer 9e663a09696c422e89edf6bd7925051e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:16.551904 28565 consensus_queue.cc:1035] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [LEADER]: Connected to new peer: Peer: permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:16.552002 28602 consensus_queue.cc:1035] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:16.552520 28568 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: TxnStatusTablet state changed. Reason: New leader 9e663a09696c422e89edf6bd7925051e. Latest consensus state: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.552613 28568 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:16.552606 28570 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: TxnStatusTablet state changed. Reason: New leader 9e663a09696c422e89edf6bd7925051e. Latest consensus state: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.552680 28570 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:16.553238 28565 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.553326 28565 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:05:16.553628 28575 mvcc.cc:205] Tried to move back new op lower bound from 7334742902993223680 to 7334742902954934272. Current Snapshot: MvccSnapshot[applied={T|T < 7334742902993223680}]
I20260929 19:05:16.553777 28568 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.553915 28568 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P 945e3912876b48df9daebf56e23eb130: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:16.554370 28602 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.554466 28602 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P 9e663a09696c422e89edf6bd7925051e: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:05:16.555006 28570 tablet_replica.cc:439] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e663a09696c422e89edf6bd7925051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 } } peers { permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 } } peers { permanent_uuid: "a3c29510c5564be092d7acc6d9a2e762" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44119 } } }
I20260929 19:05:16.555034 28573 mvcc.cc:205] Tried to move back new op lower bound from 7334742902993223680 to 7334742902954934272. Current Snapshot: MvccSnapshot[applied={T|T < 7334742902993223680 or (T in {7334742902993223680})}]
I20260929 19:05:16.555094 28570 tablet_replica.cc:442] T dfd999e4ae754b949c92f1f124d0e11e P a3c29510c5564be092d7acc6d9a2e762: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:16.562165 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.559733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:58943 (local address 127.26.78.193:36059)
0929 19:05:16.559805 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.559807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.559816 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:16.559840 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.559841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.559851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.559924 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:16.560066 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.560644 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.561425 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.561622 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.561706 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.561913 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:16.561953 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:16.562008 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.562078 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.562096 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:16.562206 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.559264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36059 (local address 127.26.78.195:58943)
0929 19:05:16.559452 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.559466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.559547 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.559943 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.559944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.559946 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:16.560012 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.560014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.560732 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.560734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.561336 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.561338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.561709 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.561710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.561768 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.561793 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:16.562022 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:16.562024 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.562073 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.562123 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260929 19:05:16.564464 28351 raft_consensus.cc:1261] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Refusing update from remote peer a3c29510c5564be092d7acc6d9a2e762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:16.564504 28211 raft_consensus.cc:1261] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Refusing update from remote peer a3c29510c5564be092d7acc6d9a2e762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:16.564662 28566 consensus_queue.cc:1035] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945e3912876b48df9daebf56e23eb130" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:16.564800 28570 consensus_queue.cc:1035] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e663a09696c422e89edf6bd7925051e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:16.573484 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.570418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:44119 (local address 127.26.78.193:48291)
0929 19:05:16.570606 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:16.570618 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:16.570669 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:16.570806 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:16.570807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:16.570808 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:16.570876 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.570877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.571796 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.571797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.572649 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:16.572651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.573106 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.573106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:16.573154 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.573167 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:16.573344 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:16.573346 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:16.573381 (+ 35us) client_negotiation.cc:240] Negotiation successful
0929 19:05:16.573413 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260929 19:05:16.573479 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.570608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:48291 (local address 127.26.78.195:44119)
0929 19:05:16.570650 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.570652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.570660 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:16.570733 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.570734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.570744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.570805 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:16.570931 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.571682 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.572720 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.573027 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.573103 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.573241 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:16.573277 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:16.573337 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.573385 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.573408 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:16.582069 28052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:16.582190 28052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:16.582861 28052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fdae13cdc20493e838da4c08233c9e4: Sending DeleteTablet for 3 replicas of tablet 0b763519e8a54cfc92721d4cc9d5c19e
I20260929 19:05:16.583113 28471 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b763519e8a54cfc92721d4cc9d5c19e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:16 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:51820
I20260929 19:05:16.583138 28191 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b763519e8a54cfc92721d4cc9d5c19e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:16 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:56106
I20260929 19:05:16.583215 28583 tablet_replica.cc:330] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: stopping tablet replica
I20260929 19:05:16.583223 28585 tablet_replica.cc:330] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: stopping tablet replica
I20260929 19:05:16.583261 28585 raft_consensus.cc:2229] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:16.583266 28583 raft_consensus.cc:2229] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:16.583292 28585 raft_consensus.cc:2258] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:16.583321 28583 raft_consensus.cc:2258] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:16.583501 28331 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b763519e8a54cfc92721d4cc9d5c19e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:16 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:43708
I20260929 19:05:16.583626 28584 tablet_replica.cc:330] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: stopping tablet replica
I20260929 19:05:16.583622 28583 ts_tablet_manager.cc:1905] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:16.583681 28584 raft_consensus.cc:2229] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:16.583721 28584 raft_consensus.cc:2258] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:16.583948 28585 ts_tablet_manager.cc:1905] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:16.585353 28583 ts_tablet_manager.cc:1918] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:16.585350 28585 ts_tablet_manager.cc:1918] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:16.585430 28585 log.cc:1196] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/0b763519e8a54cfc92721d4cc9d5c19e
I20260929 19:05:16.585431 28583 log.cc:1196] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/0b763519e8a54cfc92721d4cc9d5c19e
I20260929 19:05:16.585681 28583 ts_tablet_manager.cc:1939] T 0b763519e8a54cfc92721d4cc9d5c19e P a3c29510c5564be092d7acc6d9a2e762: Deleting consensus metadata
I20260929 19:05:16.585698 28585 ts_tablet_manager.cc:1939] T 0b763519e8a54cfc92721d4cc9d5c19e P 9e663a09696c422e89edf6bd7925051e: Deleting consensus metadata
I20260929 19:05:16.586063 28040 catalog_manager.cc:4841] TS 9e663a09696c422e89edf6bd7925051e (127.26.78.193:36059): tablet 0b763519e8a54cfc92721d4cc9d5c19e (table test-table [id=ed199597ba594b879ed2ba841c146121]) successfully deleted
I20260929 19:05:16.586081 28039 catalog_manager.cc:4841] TS a3c29510c5564be092d7acc6d9a2e762 (127.26.78.195:44119): tablet 0b763519e8a54cfc92721d4cc9d5c19e (table test-table [id=ed199597ba594b879ed2ba841c146121]) successfully deleted
I20260929 19:05:16.586287 28584 ts_tablet_manager.cc:1905] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:16.587428 28584 ts_tablet_manager.cc:1918] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:16.587496 28584 log.cc:1196] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/0b763519e8a54cfc92721d4cc9d5c19e
I20260929 19:05:16.587713 28584 ts_tablet_manager.cc:1939] T 0b763519e8a54cfc92721d4cc9d5c19e P 945e3912876b48df9daebf56e23eb130: Deleting consensus metadata
I20260929 19:05:16.588007 28040 catalog_manager.cc:4841] TS 945e3912876b48df9daebf56e23eb130 (127.26.78.194:41917): tablet 0b763519e8a54cfc92721d4cc9d5c19e (table test-table [id=ed199597ba594b879ed2ba841c146121]) successfully deleted
Sep 29 19:05:16 dist-test-slave-xz51 krb5kdc[28009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708715, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:16.595743 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.585895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56124 (local address 127.26.78.193:36059)
0929 19:05:16.585935 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.585937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.587457 (+ 1520us) server_negotiation.cc:407] Connection header received
0929 19:05:16.587588 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.587589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.587600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.587659 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:16.588393 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.588877 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.589595 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.589804 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.589908 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.592754 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:16.592762 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:16.592763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:16.592770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:16.594525 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:16.594909 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:16.594913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:16.594914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:16.594974 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:16.595174 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:16.595175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:16.595175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:16.595320 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:16.595471 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.595546 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.595623 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 19:05:16.596007 28232 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56124
I20260929 19:05:16.603456 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:16.596951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.26.78.254:46561)
0929 19:05:16.596998 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:16.597000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:16.597036 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:05:16.597107 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:16.597108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:16.597116 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:16.597154 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:16.597774 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.598343 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.599390 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:16.599582 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:16.599638 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:16.600926 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:16.600928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:16.600928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:16.600934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:16.602380 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:16.602750 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:16.602751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:16.602752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:16.602808 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:16.602998 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:16.602998 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:16.602999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:16.603096 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:16.603171 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:16.603326 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 19:05:16.603392 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 19:05:16.603734 28052 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41830
I20260929 19:05:16.604524 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 28126
I20260929 19:05:16.609891 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 28266
I20260929 19:05:16.615069 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 28406
I20260929 19:05:16.620599 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 28022
2026-09-29T19:05:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2515 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:16 dist-test-slave-xz51 krb5kdc[28625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:16 dist-test-slave-xz51 krb5kdc[28625](info): set up 2 sockets
Sep 29 19:05:16 dist-test-slave-xz51 krb5kdc[28625](info): commencing operation
krb5kdc: starting...
W20260929 19:05:17.701552 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:17 dist-test-slave-xz51 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:17.820425 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40375
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:40375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:17.901332 28638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:17.901595 28638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:17.901633 28638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:17.902841 28638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:17.902915 28638 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 19:05:17.902937 28638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:17.902959 28638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:17.902992 28638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:17.903024 28638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:17.904531 28638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:40375
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:17.904855 28638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:17.905095 28638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:17.910032 28643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:17.910020 28644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:17.910023 28638 server_base.cc:1034] running on GCE node
W20260929 19:05:17.910084 28646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:17.910427 28638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:17.910684 28638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:17.911859 28638 hybrid_clock.cc:648] HybridClock initialized: now 1790708717911841 us; error 44 us; skew 500 ppm
Sep 29 19:05:17 dist-test-slave-xz51 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708717, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:17.920837 28638 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:17.921329 28638 webserver.cc:460] Webserver started at http://127.26.78.254:37535/ using document root <none> and password file <none>
I20260929 19:05:17.921586 28638 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:17.921658 28638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:17.921809 28638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:17.923081 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "afce3a3a682f45c4a0a25029637ff3d6"
format_stamp: "Formatted at 2026-09-29 19:05:17 on dist-test-slave-xz51"
server_key: "d1ce28ed56f6173de17faf00728c8eda"
server_key_iv: "2d7ee57c2eaca0db807fd6622d18d040"
server_key_version: "encryptionkey@0"
I20260929 19:05:17.923630 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "afce3a3a682f45c4a0a25029637ff3d6"
format_stamp: "Formatted at 2026-09-29 19:05:17 on dist-test-slave-xz51"
server_key: "d1ce28ed56f6173de17faf00728c8eda"
server_key_iv: "2d7ee57c2eaca0db807fd6622d18d040"
server_key_version: "encryptionkey@0"
I20260929 19:05:17.925220 28638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:17.926182 28653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:17.926396 28638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:17.926522 28638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "afce3a3a682f45c4a0a25029637ff3d6"
format_stamp: "Formatted at 2026-09-29 19:05:17 on dist-test-slave-xz51"
server_key: "d1ce28ed56f6173de17faf00728c8eda"
server_key_iv: "2d7ee57c2eaca0db807fd6622d18d040"
server_key_version: "encryptionkey@0"
I20260929 19:05:17.926609 28638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:17.946513 28638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:17.949029 28638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:17.949210 28638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:17.953879 28638 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:40375
I20260929 19:05:17.953933 28715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:40375 every 8 connection(s)
I20260929 19:05:17.954263 28638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:17.954895 28716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:17.955552 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 28638
I20260929 19:05:17.955639 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:17.955803 26939 external_mini_cluster.cc:1428] Setting key fbe402c77cdc3d17cb55852a58a6a4f0
I20260929 19:05:17.957552 28716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Bootstrap starting.
I20260929 19:05:17.958173 28716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:17.958459 28716 log.cc:824] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:17.959146 28716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: No bootstrap required, opened a new log
I20260929 19:05:17.960455 28716 raft_consensus.cc:348] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afce3a3a682f45c4a0a25029637ff3d6" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40375 } }
I20260929 19:05:17.960614 28716 raft_consensus.cc:374] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:17.960675 28716 raft_consensus.cc:729] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afce3a3a682f45c4a0a25029637ff3d6, State: Initialized, Role: FOLLOWER
I20260929 19:05:17.960814 28716 consensus_queue.cc:260] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afce3a3a682f45c4a0a25029637ff3d6" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40375 } }
I20260929 19:05:17.960899 28716 raft_consensus.cc:388] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:17.960953 28716 raft_consensus.cc:482] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:17.961004 28716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:17.961522 28716 raft_consensus.cc:504] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afce3a3a682f45c4a0a25029637ff3d6" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40375 } }
I20260929 19:05:17.961606 28716 leader_election.cc:302] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afce3a3a682f45c4a0a25029637ff3d6; no voters:
I20260929 19:05:17.961812 28716 leader_election.cc:288] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:17.961845 28721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:17.961975 28721 raft_consensus.cc:686] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [term 1 LEADER]: Becoming Leader. State: Replica: afce3a3a682f45c4a0a25029637ff3d6, State: Running, Role: LEADER
I20260929 19:05:17.962055 28721 consensus_queue.cc:237] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afce3a3a682f45c4a0a25029637ff3d6" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40375 } }
I20260929 19:05:17.962111 28716 sys_catalog.cc:564] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:17.962375 28721 sys_catalog.cc:455] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afce3a3a682f45c4a0a25029637ff3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afce3a3a682f45c4a0a25029637ff3d6" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40375 } } }
I20260929 19:05:17.962448 28721 sys_catalog.cc:458] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:17.962563 28721 sys_catalog.cc:455] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afce3a3a682f45c4a0a25029637ff3d6. Latest consensus state: current_term: 1 leader_uuid: "afce3a3a682f45c4a0a25029637ff3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afce3a3a682f45c4a0a25029637ff3d6" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40375 } } }
I20260929 19:05:17.962705 28721 sys_catalog.cc:458] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:17.962881 28725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 29 19:05:17 dist-test-slave-xz51 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:17.963394 28725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:17.966930 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:17.956337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.26.78.254:40375)
0929 19:05:17.956524 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:17.956535 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:17.956568 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:05:17.956968 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:17.956975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:17.957022 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:17.957171 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:17.957902 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:17.959017 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:17.959826 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:17.960568 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:17.960695 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:17.963559 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:17.963563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:17.963564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:17.963577 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:17.965316 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:17.965696 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:17.965700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:17.965700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:17.965803 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:17.966068 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:17.966069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:17.966069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:17.966320 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:17.966492 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:17.966599 (+ 107us) server_negotiation.cc:299] Negotiation successful
0929 19:05:17.966742 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260929 19:05:17.968139 28725 catalog_manager.cc:1347] Generated new cluster ID: e949b8c9c5424785be01a4e9d9e9c0e3
I20260929 19:05:17.968199 28725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:17.978375 28725 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:17.979087 28725 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:17.989917 28725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Generated new TSK 0
I20260929 19:05:17.990180 28725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:18.024914 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40375
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:18.111847 28742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:18.112067 28742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:18.112094 28742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:18.113240 28742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:18.113279 28742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:18.113303 28742 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 19:05:18.113315 28742 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:05:18.113332 28742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:18.114873 28742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:18.115082 28742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:18.115273 28742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:18.120342 28747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:18.120342 28750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:18.120395 28748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:18.120565 28742 server_base.cc:1034] running on GCE node
I20260929 19:05:18.120764 28742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:18.121037 28742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:18.122176 28742 hybrid_clock.cc:648] HybridClock initialized: now 1790708718122159 us; error 34 us; skew 500 ppm
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:18.124658 28742 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:18.125109 28742 webserver.cc:460] Webserver started at http://127.26.78.193:38135/ using document root <none> and password file <none>
I20260929 19:05:18.125355 28742 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:18.125402 28742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:18.125569 28742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:18.126686 28742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "5a272539b0f54f8d89f26435eb66ccfe"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "9cd760768e878675628d7395018a96a3"
server_key_iv: "6b2a505611219d67279c95e68a165170"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.127040 28742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "5a272539b0f54f8d89f26435eb66ccfe"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "9cd760768e878675628d7395018a96a3"
server_key_iv: "6b2a505611219d67279c95e68a165170"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.128341 28742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:05:18.129068 28757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.129295 28742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:18.129396 28742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "5a272539b0f54f8d89f26435eb66ccfe"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "9cd760768e878675628d7395018a96a3"
server_key_iv: "6b2a505611219d67279c95e68a165170"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.129453 28742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:18.143551 28742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:18.145980 28742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:18.146099 28742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:18.146610 28742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:18.146706 28742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.146759 28742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:18.146822 28742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.153528 28742 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:35181
I20260929 19:05:18.153595 28872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:35181 every 8 connection(s)
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:18.153990 28742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:05:18.157212 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.147140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:60231 (local address 127.26.78.254:40375)
0929 19:05:18.147213 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.147216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.147490 (+ 274us) server_negotiation.cc:407] Connection header received
0929 19:05:18.147996 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.147998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.148013 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.148101 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.148972 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.149788 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.151445 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.151798 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.151890 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.154551 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.154552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.154552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.154559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.156053 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.156418 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.156419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.156420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.156471 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.156683 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.156684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.156684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.156788 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.156839 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.157067 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.157146 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:05:18.157403 28766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.147135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40375 (local address 127.26.78.193:60231)
0929 19:05:18.147335 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.147393 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.147897 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.148181 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.148185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.148564 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:18.148882 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.148889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.149901 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.149902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.151318 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.151321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.151891 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.151892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.152001 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.152601 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:18.152611 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:18.154458 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:18.156126 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.156131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.156133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.156352 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.156533 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.156533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.156534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.156629 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.156843 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:18.156847 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:18.156960 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.157042 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.157161 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":130,"threads_started":1}
I20260929 19:05:18.157948 28873 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40375
I20260929 19:05:18.158035 28873 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:18.158223 28873 heartbeater.cc:511] Master 127.26.78.254:40375 requested a full tablet report, sending...
I20260929 19:05:18.158810 28669 ts_manager.cc:194] Registered new tserver with Master: 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181)
I20260929 19:05:18.159750 28669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:60231
I20260929 19:05:18.160516 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 28742
I20260929 19:05:18.160619 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:18.160805 26939 external_mini_cluster.cc:1428] Setting key b6fd4a5ca4adac5f48a759bf2ba0bc89
I20260929 19:05:18.165355 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.159121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:58461 (local address 127.26.78.254:40375)
0929 19:05:18.159164 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.159166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.159240 (+ 74us) server_negotiation.cc:407] Connection header received
0929 19:05:18.159319 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.159320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.159328 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.159367 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.159686 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.160286 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.161184 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.161480 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.161549 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.162593 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.162595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.162595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.162602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.164126 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.164548 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.164549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.164550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.164607 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.164823 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.164824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.164825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.164942 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.165020 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.165184 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.165263 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:05:18.165377 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.159037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40375 (local address 127.26.78.193:58461)
0929 19:05:18.159183 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.159194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.159255 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.159379 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.159380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.159556 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:18.159624 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.159625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.160394 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.160395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.161103 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.161106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.161561 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.161562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.161641 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.162128 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:18.162134 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:18.162525 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:18.164245 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.164246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.164247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.164468 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.164689 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.164690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.164690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.164755 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.165011 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:18.165012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:18.165070 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.165158 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.165265 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:18.187644 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40375
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:18.268671 28882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:18.268895 28882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:18.268920 28882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:18.270044 28882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:18.270083 28882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:18.270103 28882 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 19:05:18.270115 28882 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:05:18.270133 28882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:18.271589 28882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:18.271963 28882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:18.272218 28882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:18.277361 28888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:18.277374 28887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:18.277374 28890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:18.277534 28882 server_base.cc:1034] running on GCE node
I20260929 19:05:18.277792 28882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:18.278131 28882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:18.279312 28882 hybrid_clock.cc:648] HybridClock initialized: now 1790708718279297 us; error 27 us; skew 500 ppm
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:18.282177 28882 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:18.282588 28882 webserver.cc:460] Webserver started at http://127.26.78.194:39127/ using document root <none> and password file <none>
I20260929 19:05:18.282791 28882 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:18.282835 28882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:18.282923 28882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:18.283936 28882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "b891b380fa19460ead53c4b862fa1cd1"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "6f270d7f155b35cc0a83ab827d217e90"
server_key_iv: "99db2128377f482cb5b7aeddd7c81f94"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.284240 28882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "b891b380fa19460ead53c4b862fa1cd1"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "6f270d7f155b35cc0a83ab827d217e90"
server_key_iv: "99db2128377f482cb5b7aeddd7c81f94"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.285544 28882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:18.286375 28897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.286528 28882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.286619 28882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "b891b380fa19460ead53c4b862fa1cd1"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "6f270d7f155b35cc0a83ab827d217e90"
server_key_iv: "99db2128377f482cb5b7aeddd7c81f94"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.286675 28882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:18.303654 28882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:18.305529 28882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:18.305641 28882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:18.306174 28882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:18.306219 28882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.306245 28882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:18.306298 28882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.313275 28882 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39487
I20260929 19:05:18.313346 29012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39487 every 8 connection(s)
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:18.313732 28882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:18.316468 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.306689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:33659 (local address 127.26.78.254:40375)
0929 19:05:18.306737 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.306739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.306976 (+ 237us) server_negotiation.cc:407] Connection header received
0929 19:05:18.307561 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.307563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.307579 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.307665 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.308553 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.309398 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.310671 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.311026 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.311116 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.314028 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.314030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.314031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.314040 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.315358 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.315710 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.315710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.315711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.315760 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.315978 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.315978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.315978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.316078 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.316127 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.316348 (+ 221us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.316406 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:18.316727 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.306681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40375 (local address 127.26.78.194:33659)
0929 19:05:18.306858 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.306899 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.307410 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.307753 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.307757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.308131 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:18.308443 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.308450 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.309496 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.309497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.310570 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.310572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.311185 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.311187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.311361 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.311892 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:18.311901 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:18.313905 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:18.315425 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.315431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.315432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.315643 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.315831 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.315831 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.315832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.315924 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.316131 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:18.316135 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:18.316242 (+ 107us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.316328 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.316445 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260929 19:05:18.317200 29013 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40375
I20260929 19:05:18.317286 29013 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:18.317476 29013 heartbeater.cc:511] Master 127.26.78.254:40375 requested a full tablet report, sending...
I20260929 19:05:18.317874 28670 ts_manager.cc:194] Registered new tserver with Master: b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487)
I20260929 19:05:18.318249 28670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:33659
I20260929 19:05:18.323043 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 28882
I20260929 19:05:18.323148 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:18.323372 26939 external_mini_cluster.cc:1428] Setting key 450d27553f711fe620a981a8570b54ba
I20260929 19:05:18.323674 29019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.318315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40375 (local address 127.26.78.194:55601)
0929 19:05:18.318442 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.318454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.318510 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.318671 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.318672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.318827 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:18.318885 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.318886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.319552 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.319552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.320047 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.320048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.320377 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.320378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.320429 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.320886 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:18.320890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:18.321183 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:18.322521 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.322522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.322522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.322678 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.322892 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.322893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.322894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.322968 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.323314 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:18.323316 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:18.323375 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.323457 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.323569 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260929 19:05:18.323688 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.318361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:55601 (local address 127.26.78.254:40375)
0929 19:05:18.318397 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.318399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.318504 (+ 105us) server_negotiation.cc:407] Connection header received
0929 19:05:18.318583 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.318584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.318595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.318659 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.318956 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.319466 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.320115 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.320314 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.320365 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.321252 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.321253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.321253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.321258 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.322449 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.322741 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.322742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.322742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.322791 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.323036 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.323037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.323037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.323231 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.323314 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.323498 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.323578 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:18.348570 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40375
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:18.428930 29022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:18.429139 29022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:18.429165 29022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:18.430375 29022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:18.430421 29022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:18.430442 29022 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 19:05:18.430454 29022 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:05:18.430471 29022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:18.431905 29022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:18.432087 29022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:18.432262 29022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:18.437332 29030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:18.437352 29028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:18.437340 29027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:18.437707 29022 server_base.cc:1034] running on GCE node
I20260929 19:05:18.437944 29022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:18.438179 29022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:18.439325 29022 hybrid_clock.cc:648] HybridClock initialized: now 1790708718439319 us; error 35 us; skew 500 ppm
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:18.441776 29022 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:18.442216 29022 webserver.cc:460] Webserver started at http://127.26.78.195:32839/ using document root <none> and password file <none>
I20260929 19:05:18.442406 29022 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:18.442447 29022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:18.442531 29022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:18.443540 29022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "78fb1171e86c4f16928230d1cc43fdfe"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "4adbdde32f863b53bccc8b80861de3f3"
server_key_iv: "b191701b5cd04090a94fe7ab365d8ff6"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.443840 29022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "78fb1171e86c4f16928230d1cc43fdfe"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "4adbdde32f863b53bccc8b80861de3f3"
server_key_iv: "b191701b5cd04090a94fe7ab365d8ff6"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.445057 29022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:18.445719 29037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.445886 29022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:18.445976 29022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "78fb1171e86c4f16928230d1cc43fdfe"
format_stamp: "Formatted at 2026-09-29 19:05:18 on dist-test-slave-xz51"
server_key: "4adbdde32f863b53bccc8b80861de3f3"
server_key_iv: "b191701b5cd04090a94fe7ab365d8ff6"
server_key_version: "encryptionkey@0"
I20260929 19:05:18.446025 29022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:18.452674 29022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:18.454550 29022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:18.454712 29022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:18.455194 29022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:18.455240 29022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.455262 29022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:18.455307 29022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.462036 29022 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:36255
I20260929 19:05:18.462103 29152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:36255 every 8 connection(s)
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:18.462551 29022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:05:18.463204 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 29022
I20260929 19:05:18.463330 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:18.463532 26939 external_mini_cluster.cc:1428] Setting key 60f1f7c905ac117996e6a1aaac37c9d9
I20260929 19:05:18.465776 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.455698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:57607 (local address 127.26.78.254:40375)
0929 19:05:18.455766 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.455769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.455926 (+ 157us) server_negotiation.cc:407] Connection header received
0929 19:05:18.456402 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.456404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.456418 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.456501 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.457315 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.458202 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.459791 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.460140 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.460224 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.462898 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.462900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.462901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.462909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.464620 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.465000 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.465001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.465001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.465052 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.465226 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.465227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.465227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.465350 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.465399 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.465641 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.465714 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:05:18.465903 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.455655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40375 (local address 127.26.78.195:57607)
0929 19:05:18.455802 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.455840 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.456308 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.456589 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.456593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.456930 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:18.457206 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.457212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.458319 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.458320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.459685 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.459687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.460233 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.460234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.460375 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.460915 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:18.460927 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:18.462786 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:18.464706 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.464712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.464714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.464937 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.465095 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.465096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.465096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.465178 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.465390 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:18.465392 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:18.465495 (+ 103us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.465575 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.465689 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260929 19:05:18.466493 29153 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40375
I20260929 19:05:18.466583 29153 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:18.466822 29153 heartbeater.cc:511] Master 127.26.78.254:40375 requested a full tablet report, sending...
I20260929 19:05:18.467242 28670 ts_manager.cc:194] Registered new tserver with Master: 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255)
I20260929 19:05:18.467617 28670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:57607
I20260929 19:05:18.472895 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.467702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:49877 (local address 127.26.78.254:40375)
0929 19:05:18.467744 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.467747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.467862 (+ 115us) server_negotiation.cc:407] Connection header received
0929 19:05:18.467937 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.467938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.467948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.467998 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.468318 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.468829 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.469535 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.469759 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.469814 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.470754 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.470755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.470755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.470761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.471997 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.472305 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.472306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.472306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.472354 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.472513 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.472513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.472513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.472604 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.472705 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.472775 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.472832 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:05:18.472904 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.467639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40375 (local address 127.26.78.195:49877)
0929 19:05:18.467802 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.467812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.467874 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.468017 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.468018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.468176 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:18.468232 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.468233 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.468949 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.468950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.469457 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.469459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.469894 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.469894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.469971 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.470343 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:18.470347 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:18.470629 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:18.472073 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.472074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.472074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.472241 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.472410 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.472410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.472411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.472462 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.472657 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:18.472657 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:18.472703 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.472750 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.472834 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260929 19:05:18.475507 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:18.498746 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.491365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.26.78.254:40375)
0929 19:05:18.491429 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.491431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.491441 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:18.491472 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.491473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.491484 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.491552 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.492228 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.493122 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.494009 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.494227 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.494309 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.496578 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.496579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.496580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.496587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.497744 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.498115 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.498116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.498116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.498168 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.498346 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.498346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.498347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.498449 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.498524 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.498622 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.498680 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:05:18.500370 28670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:18.501968 28670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:18.508441 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.504277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:35181 (local address 127.0.0.1:54824)
0929 19:05:18.504634 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.504643 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.504683 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.505025 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.505026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.505029 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:18.505082 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.505084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.506394 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.506395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.507440 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.507442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.508120 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.508121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.508198 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.508212 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.508296 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.508348 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":91,"threads_started":1}
I20260929 19:05:18.508560 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.504433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.26.78.194:39487)
0929 19:05:18.504632 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.504637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.504673 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:05:18.504753 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.504755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.504774 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.504873 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:18.505069 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.505957 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.507482 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.508214 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.508306 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.508365 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.508421 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.508464 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260929 19:05:18.508628 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.504689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.26.78.193:35181)
0929 19:05:18.504869 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.504874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.504883 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:18.504920 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.504922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.504937 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.505013 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:18.505167 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.506290 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.507526 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.508026 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.508121 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.508183 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.508484 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.508527 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260929 19:05:18.508556 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.504267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:36255 (local address 127.0.0.1:35688)
0929 19:05:18.504423 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.504464 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.504532 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.505174 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.505175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.505176 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:18.505216 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.505217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.506486 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.506487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.507460 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.507463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.508340 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.508341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.508410 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.508414 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.508445 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.508485 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260929 19:05:18.508911 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.504504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.26.78.195:36255)
0929 19:05:18.504894 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.504898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.504907 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:18.504944 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.504946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.504960 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.505302 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:18.505358 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.506407 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.507555 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.508247 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.508633 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.508705 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.508732 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.508772 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":78,"threads_started":1}
I20260929 19:05:18.508827 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.504268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39487 (local address 127.0.0.1:50010)
0929 19:05:18.504611 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.504622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.504681 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.504901 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.504904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.504906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:18.504982 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.504984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.506048 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.506049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.507389 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.507392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.508293 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.508294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.508346 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.508348 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.508404 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.508436 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":58,"threads_started":1}
I20260929 19:05:18.509505 28807 tablet_service.cc:1467] Processing CreateTablet for tablet 8657358fe2854ca0a09209f1c98d13c8 (DEFAULT_TABLE table=test-table [id=5ba8370306f74f9da93690fafc52525b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:18.509581 28947 tablet_service.cc:1467] Processing CreateTablet for tablet 8657358fe2854ca0a09209f1c98d13c8 (DEFAULT_TABLE table=test-table [id=5ba8370306f74f9da93690fafc52525b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:18.509575 29087 tablet_service.cc:1467] Processing CreateTablet for tablet 8657358fe2854ca0a09209f1c98d13c8 (DEFAULT_TABLE table=test-table [id=5ba8370306f74f9da93690fafc52525b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:18.509912 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8657358fe2854ca0a09209f1c98d13c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.509912 28947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8657358fe2854ca0a09209f1c98d13c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.509948 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8657358fe2854ca0a09209f1c98d13c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.512470 29176 tablet_bootstrap.cc:492] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Bootstrap starting.
I20260929 19:05:18.513298 29176 tablet_bootstrap.cc:654] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.513652 29178 tablet_bootstrap.cc:492] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Bootstrap starting.
I20260929 19:05:18.513685 29176 log.cc:824] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:18.514307 29178 tablet_bootstrap.cc:654] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.514497 29176 tablet_bootstrap.cc:492] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: No bootstrap required, opened a new log
I20260929 19:05:18.514576 29176 ts_tablet_manager.cc:1397] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:05:18.514657 29178 log.cc:824] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:18.514986 29177 tablet_bootstrap.cc:492] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Bootstrap starting.
I20260929 19:05:18.515491 29177 tablet_bootstrap.cc:654] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.515843 29177 log.cc:824] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:18.516403 29178 tablet_bootstrap.cc:492] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: No bootstrap required, opened a new log
I20260929 19:05:18.516486 29178 ts_tablet_manager.cc:1397] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 19:05:18.516366 29176 raft_consensus.cc:348] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.516527 29176 raft_consensus.cc:374] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.516594 29177 tablet_bootstrap.cc:492] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: No bootstrap required, opened a new log
I20260929 19:05:18.516609 29176 raft_consensus.cc:729] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb1171e86c4f16928230d1cc43fdfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.516661 29177 ts_tablet_manager.cc:1397] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:18.516776 29176 consensus_queue.cc:260] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.517084 29153 heartbeater.cc:503] Master 127.26.78.254:40375 was elected leader, sending a full tablet report...
I20260929 19:05:18.517210 29176 ts_tablet_manager.cc:1428] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:05:18.518566 29178 raft_consensus.cc:348] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.518714 29178 raft_consensus.cc:374] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.518759 29178 raft_consensus.cc:729] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a272539b0f54f8d89f26435eb66ccfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.518872 29178 consensus_queue.cc:260] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.518946 29177 raft_consensus.cc:348] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.519127 29177 raft_consensus.cc:374] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.519176 29177 raft_consensus.cc:729] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b891b380fa19460ead53c4b862fa1cd1, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.519193 28873 heartbeater.cc:503] Master 127.26.78.254:40375 was elected leader, sending a full tablet report...
I20260929 19:05:18.519294 29177 consensus_queue.cc:260] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.519176 29178 ts_tablet_manager.cc:1428] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:05:18.519558 29177 ts_tablet_manager.cc:1428] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:05:18.519699 29013 heartbeater.cc:503] Master 127.26.78.254:40375 was elected leader, sending a full tablet report...
W20260929 19:05:18.564455 29014 tablet.cc:2369] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:05:18.572677 29184 raft_consensus.cc:482] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:18.572855 29184 raft_consensus.cc:504] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.573246 29184 leader_election.cc:288] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255), 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181)
I20260929 19:05:18.576356 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.573838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:56659 (local address 127.26.78.195:36255)
0929 19:05:18.573891 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.573892 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.573901 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:18.573926 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.573927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.573937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.573992 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:18.574194 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.574680 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.575609 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.575972 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.576029 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.576035 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.576200 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.576247 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:05:18.576352 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.573532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:36255 (local address 127.26.78.194:56659)
0929 19:05:18.573716 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.573725 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.573764 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.574026 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.574027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.574029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:18.574107 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.574109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.574757 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.574758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.575521 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.575523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.576051 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.576052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.576133 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.576137 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.576179 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.576218 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":19,"thread_start_us":114,"threads_started":1}
I20260929 19:05:18.576370 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.573525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:42583 (local address 127.26.78.193:35181)
0929 19:05:18.573573 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.573575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.573586 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:05:18.573670 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.573671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.573678 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.573750 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:18.574021 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.574584 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.575619 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.575990 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.576077 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.576085 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.576271 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.576307 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:05:18.576366 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.573472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:35181 (local address 127.26.78.194:42583)
0929 19:05:18.573530 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.573545 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.573597 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.573756 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.573757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.573759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:18.573873 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.573876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.574661 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.574662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.575518 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.575520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.576088 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.576089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.576193 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.576197 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.576253 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.576287 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":38}
I20260929 19:05:18.576694 29107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8657358fe2854ca0a09209f1c98d13c8" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb1171e86c4f16928230d1cc43fdfe" is_pre_election: true
I20260929 19:05:18.576753 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8657358fe2854ca0a09209f1c98d13c8" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe" is_pre_election: true
I20260929 19:05:18.576869 29107 raft_consensus.cc:2454] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 0.
I20260929 19:05:18.576942 28827 raft_consensus.cc:2454] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 0.
I20260929 19:05:18.577127 28898 leader_election.cc:302] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb1171e86c4f16928230d1cc43fdfe, b891b380fa19460ead53c4b862fa1cd1; no voters:
I20260929 19:05:18.577304 29184 raft_consensus.cc:2781] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:18.577370 29184 raft_consensus.cc:482] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:18.577404 29184 raft_consensus.cc:3037] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.578379 29184 raft_consensus.cc:504] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.578528 29184 leader_election.cc:288] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255), 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181)
I20260929 19:05:18.578708 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8657358fe2854ca0a09209f1c98d13c8" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe"
I20260929 19:05:18.578723 29107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8657358fe2854ca0a09209f1c98d13c8" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb1171e86c4f16928230d1cc43fdfe"
I20260929 19:05:18.578792 28827 raft_consensus.cc:3037] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.578815 29107 raft_consensus.cc:3037] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.579509 28827 raft_consensus.cc:2454] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 1.
I20260929 19:05:18.579524 29107 raft_consensus.cc:2454] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 1.
I20260929 19:05:18.579687 28898 leader_election.cc:302] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb1171e86c4f16928230d1cc43fdfe, b891b380fa19460ead53c4b862fa1cd1; no voters:
I20260929 19:05:18.579842 29184 raft_consensus.cc:2781] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:18.579986 29184 raft_consensus.cc:686] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 1 LEADER]: Becoming Leader. State: Replica: b891b380fa19460ead53c4b862fa1cd1, State: Running, Role: LEADER
I20260929 19:05:18.580080 29184 consensus_queue.cc:237] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } }
I20260929 19:05:18.580969 28669 catalog_manager.cc:5495] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "b891b380fa19460ead53c4b862fa1cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } health_report { overall_health: UNKNOWN } } }
Sep 29 19:05:18 dist-test-slave-xz51 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708717, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:18.633487 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.626641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38362 (local address 127.26.78.254:40375)
0929 19:05:18.626699 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.626701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.626887 (+ 186us) server_negotiation.cc:407] Connection header received
0929 19:05:18.626979 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.626979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.626987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.627047 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:18.627532 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.628024 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.628765 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.628965 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.629040 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.631041 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:18.631042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:18.631043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:18.631049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:18.632268 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.632645 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.632646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.632647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.632722 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:18.632949 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:18.632950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:18.632950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:18.633060 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:18.633139 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.633292 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.633375 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:05:18.633555 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.626538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40375 (local address 127.0.0.1:38362)
0929 19:05:18.626802 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.626813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.626907 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.627052 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.627053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.627397 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:18.627443 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.627446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.628118 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.628119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.628678 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.628680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.629039 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.629040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.629092 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.629574 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:18.629579 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:18.630951 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:18.632363 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.632370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.632373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.632559 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.632821 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:18.632822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:18.632822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:18.632883 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:18.633129 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:18.633136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:18.633186 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.633270 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.633431 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":49,"thread_start_us":169,"threads_started":1}
I20260929 19:05:18.635573 28669 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:38362:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 19:05:18.635748 28669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:18.637198 28947 tablet_service.cc:1467] Processing CreateTablet for tablet 434249796cf54759bac066e98cb7efb7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3e5cc2238884ccc90c20d26dbbdf099]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:05:18.637235 29087 tablet_service.cc:1467] Processing CreateTablet for tablet 434249796cf54759bac066e98cb7efb7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3e5cc2238884ccc90c20d26dbbdf099]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:05:18.637403 28947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434249796cf54759bac066e98cb7efb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.637403 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434249796cf54759bac066e98cb7efb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.638018 28807 tablet_service.cc:1467] Processing CreateTablet for tablet 434249796cf54759bac066e98cb7efb7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3e5cc2238884ccc90c20d26dbbdf099]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:05:18.638182 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434249796cf54759bac066e98cb7efb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.638792 29177 tablet_bootstrap.cc:492] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: Bootstrap starting.
I20260929 19:05:18.639189 29177 tablet_bootstrap.cc:654] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.639477 29178 tablet_bootstrap.cc:492] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: Bootstrap starting.
I20260929 19:05:18.639508 29176 tablet_bootstrap.cc:492] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: Bootstrap starting.
I20260929 19:05:18.640004 29176 tablet_bootstrap.cc:654] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.640089 29178 tablet_bootstrap.cc:654] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.640659 29176 tablet_bootstrap.cc:492] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: No bootstrap required, opened a new log
I20260929 19:05:18.640708 29177 tablet_bootstrap.cc:492] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: No bootstrap required, opened a new log
I20260929 19:05:18.640713 29176 ts_tablet_manager.cc:1397] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:18.640755 29177 ts_tablet_manager.cc:1397] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:18.640910 29176 raft_consensus.cc:348] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.641000 29176 raft_consensus.cc:374] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.641028 29176 raft_consensus.cc:729] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb1171e86c4f16928230d1cc43fdfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.640962 29177 raft_consensus.cc:348] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.641072 29177 raft_consensus.cc:374] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.641095 29177 raft_consensus.cc:729] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b891b380fa19460ead53c4b862fa1cd1, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.641074 29176 consensus_queue.cc:260] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.641141 29177 consensus_queue.cc:260] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.641171 29176 ts_tablet_manager.cc:1428] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.641238 29177 ts_tablet_manager.cc:1428] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:18.641244 29182 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.641371 29182 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:18.641358 29184 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.641449 29184 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:18.641495 29178 tablet_bootstrap.cc:492] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: No bootstrap required, opened a new log
I20260929 19:05:18.641533 29178 ts_tablet_manager.cc:1397] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:18.641677 29178 raft_consensus.cc:348] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.641754 29178 raft_consensus.cc:374] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.641782 29178 raft_consensus.cc:729] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a272539b0f54f8d89f26435eb66ccfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.641866 29178 consensus_queue.cc:260] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.641961 29178 ts_tablet_manager.cc:1428] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.642074 29183 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.642203 29183 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: This TxnStatusTablet replica's current role is: FOLLOWER
W20260929 19:05:18.654808 28874 tablet.cc:2369] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:05:18.713198 29154 tablet.cc:2369] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:05:18.772429 29183 raft_consensus.cc:482] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:18.772541 29183 raft_consensus.cc:504] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.772946 29183 leader_election.cc:288] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487), 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255)
I20260929 19:05:18.776170 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.773299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:44497 (local address 127.26.78.195:36255)
0929 19:05:18.773347 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.773349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.773492 (+ 143us) server_negotiation.cc:407] Connection header received
0929 19:05:18.773565 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.773566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.773576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.773632 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:18.773839 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.774346 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.775458 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.775812 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.775894 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.775903 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.776057 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.776094 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:05:18.776211 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.773208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:36255 (local address 127.26.78.193:44497)
0929 19:05:18.773403 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.773412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.773504 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.773649 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.773650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.773652 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:18.773727 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.773729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.774443 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.774444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.775344 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.775346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.775905 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.775906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.775978 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.775982 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.776048 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.776089 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":114,"thread_start_us":71,"threads_started":1}
I20260929 19:05:18.776211 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.773322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:36061 (local address 127.26.78.194:39487)
0929 19:05:18.773384 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.773387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.773458 (+ 71us) server_negotiation.cc:407] Connection header received
0929 19:05:18.773564 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.773565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.773576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.773658 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:18.773862 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.774356 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.775456 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.775831 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.775923 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.775932 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.776100 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.776137 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:05:18.776235 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.773209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39487 (local address 127.26.78.193:36061)
0929 19:05:18.773370 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.773383 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.773475 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.773684 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.773685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.773688 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:18.773764 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.773766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.774460 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.774461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.775342 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.775345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.775923 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.775924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.776009 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.776012 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.776086 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.776130 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":19,"thread_start_us":71,"threads_started":1}
I20260929 19:05:18.776512 29107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434249796cf54759bac066e98cb7efb7" candidate_uuid: "5a272539b0f54f8d89f26435eb66ccfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb1171e86c4f16928230d1cc43fdfe" is_pre_election: true
I20260929 19:05:18.776628 29107 raft_consensus.cc:2454] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a272539b0f54f8d89f26435eb66ccfe in term 0.
I20260929 19:05:18.776724 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434249796cf54759bac066e98cb7efb7" candidate_uuid: "5a272539b0f54f8d89f26435eb66ccfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891b380fa19460ead53c4b862fa1cd1" is_pre_election: true
I20260929 19:05:18.776844 28967 raft_consensus.cc:2454] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a272539b0f54f8d89f26435eb66ccfe in term 0.
I20260929 19:05:18.776816 28758 leader_election.cc:302] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a272539b0f54f8d89f26435eb66ccfe, 78fb1171e86c4f16928230d1cc43fdfe; no voters:
I20260929 19:05:18.776974 29183 raft_consensus.cc:2781] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:18.777021 29183 raft_consensus.cc:482] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:18.777047 29183 raft_consensus.cc:3037] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.777729 29183 raft_consensus.cc:504] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.777907 29183 leader_election.cc:288] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [CANDIDATE]: Term 1 election: Requested vote from peers b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487), 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255)
I20260929 19:05:18.778043 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434249796cf54759bac066e98cb7efb7" candidate_uuid: "5a272539b0f54f8d89f26435eb66ccfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891b380fa19460ead53c4b862fa1cd1"
I20260929 19:05:18.778137 28967 raft_consensus.cc:3037] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.778095 29107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434249796cf54759bac066e98cb7efb7" candidate_uuid: "5a272539b0f54f8d89f26435eb66ccfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb1171e86c4f16928230d1cc43fdfe"
I20260929 19:05:18.778184 29107 raft_consensus.cc:3037] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.778825 29107 raft_consensus.cc:2454] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a272539b0f54f8d89f26435eb66ccfe in term 1.
I20260929 19:05:18.778832 28967 raft_consensus.cc:2454] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a272539b0f54f8d89f26435eb66ccfe in term 1.
I20260929 19:05:18.778997 28758 leader_election.cc:302] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a272539b0f54f8d89f26435eb66ccfe, 78fb1171e86c4f16928230d1cc43fdfe; no voters:
I20260929 19:05:18.779130 29183 raft_consensus.cc:2781] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:18.779277 29183 raft_consensus.cc:686] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 LEADER]: Becoming Leader. State: Replica: 5a272539b0f54f8d89f26435eb66ccfe, State: Running, Role: LEADER
I20260929 19:05:18.779410 29183 consensus_queue.cc:237] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.779954 29199 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: TxnStatusTablet state changed. Reason: New leader 5a272539b0f54f8d89f26435eb66ccfe. Latest consensus state: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.780045 29199 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:05:18.780265 29201 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 19:05:18.780400 29201 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 19:05:18.780262 28669 catalog_manager.cc:5495] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193). New cstate: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } health_report { overall_health: UNKNOWN } } }
I20260929 19:05:18.789287 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.786481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.26.78.193:35181)
0929 19:05:18.786518 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.786520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.786580 (+ 60us) server_negotiation.cc:407] Connection header received
0929 19:05:18.786669 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.786670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.786684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.786741 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:18.786940 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.787411 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.788501 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.788680 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.788733 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.788936 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:18.789028 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:18.789087 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.789158 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.789176 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:05:18.789287 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.786433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:35181 (local address 127.0.0.1:54826)
0929 19:05:18.786517 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.786530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.786586 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.786777 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.786778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.786780 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:18.786851 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.786854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.787494 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.787496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.788413 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.788416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.788749 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.788750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.788825 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.788843 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:18.789095 (+ 252us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:18.789104 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.789157 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.789207 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 19:05:18.791234 29107 raft_consensus.cc:1261] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Refusing update from remote peer 5a272539b0f54f8d89f26435eb66ccfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:18.791234 28967 raft_consensus.cc:1261] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Refusing update from remote peer 5a272539b0f54f8d89f26435eb66ccfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:18.791553 29199 consensus_queue.cc:1035] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:18.791673 29183 consensus_queue.cc:1035] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:18.792757 29184 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: TxnStatusTablet state changed. Reason: New leader 5a272539b0f54f8d89f26435eb66ccfe. Latest consensus state: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.792858 29184 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:18.793020 29182 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: TxnStatusTablet state changed. Reason: New leader 5a272539b0f54f8d89f26435eb66ccfe. Latest consensus state: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.793130 29182 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:18.793711 29183 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.793836 29183 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:05:18.793919 29184 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.793998 29184 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P b891b380fa19460ead53c4b862fa1cd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:18.794363 29205 mvcc.cc:205] Tried to move back new op lower bound from 7334742912166436864 to 7334742912121167872. Current Snapshot: MvccSnapshot[applied={T|T < 7334742912166436864}]
I20260929 19:05:18.795379 29199 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.795461 29199 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P 5a272539b0f54f8d89f26435eb66ccfe: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:05:18.796195 29182 tablet_replica.cc:439] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a272539b0f54f8d89f26435eb66ccfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } } }
I20260929 19:05:18.796286 29182 tablet_replica.cc:442] T 434249796cf54759bac066e98cb7efb7 P 78fb1171e86c4f16928230d1cc43fdfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:05:18.799075 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.796483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.26.78.194:39487)
0929 19:05:18.796548 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.796550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.796558 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:18.796585 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.796586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.796613 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.796687 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:18.796859 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.797401 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.798266 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.798506 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.798583 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.798751 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:18.798858 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:18.798947 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.798973 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.798994 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:18.804342 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.801562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:38991 (local address 127.26.78.193:35181)
0929 19:05:18.801602 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.801604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.801762 (+ 158us) server_negotiation.cc:407] Connection header received
0929 19:05:18.801864 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.801864 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.801874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.801935 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:18.802107 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.802592 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.803685 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.803861 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.803912 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.804123 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:18.804147 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:18.804185 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.804266 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.804288 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:18.804396 29019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.801517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:35181 (local address 127.26.78.194:38991)
0929 19:05:18.801685 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.801699 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.801763 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.801949 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.801950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.801951 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:18.802012 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.802014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.802669 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.802670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.803600 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.803602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.803934 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.803935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.804008 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.804024 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:18.804207 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:18.804209 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.804263 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.804310 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260929 19:05:18.806902 29107 raft_consensus.cc:1261] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Refusing update from remote peer b891b380fa19460ead53c4b862fa1cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:18.806957 28827 raft_consensus.cc:1261] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Refusing update from remote peer b891b380fa19460ead53c4b862fa1cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:18.807220 29186 consensus_queue.cc:1035] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:18.807339 29184 consensus_queue.cc:1035] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:18.816458 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.813788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:56861 (local address 127.26.78.194:39487)
0929 19:05:18.813853 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:18.813856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:18.814006 (+ 150us) server_negotiation.cc:407] Connection header received
0929 19:05:18.814077 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:18.814078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:18.814088 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:18.814132 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:18.814257 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.814703 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.815825 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.816004 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.816056 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.816240 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:18.816266 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:18.816306 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:18.816390 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 19:05:18.816406 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:05:18.816536 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:18.813749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39487 (local address 127.26.78.193:56861)
0929 19:05:18.813947 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:18.813959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:18.814021 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:18.814135 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:18.814136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:18.814137 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:18.814202 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.814204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.814791 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.814792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.815753 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:18.815756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:18.816078 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:18.816079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:18.816147 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:18.816162 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:18.816329 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:18.816337 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 19:05:18.816389 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:05:18.816430 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260929 19:05:18.825984 28669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:18.826123 28669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:18.826917 28669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Sending DeleteTablet for 3 replicas of tablet 8657358fe2854ca0a09209f1c98d13c8
I20260929 19:05:18.827296 28947 tablet_service.cc:1514] Processing DeleteTablet for tablet 8657358fe2854ca0a09209f1c98d13c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:18 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:50010
I20260929 19:05:18.827303 28807 tablet_service.cc:1514] Processing DeleteTablet for tablet 8657358fe2854ca0a09209f1c98d13c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:18 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:54824
I20260929 19:05:18.827728 29223 tablet_replica.cc:330] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: stopping tablet replica
I20260929 19:05:18.827739 29087 tablet_service.cc:1514] Processing DeleteTablet for tablet 8657358fe2854ca0a09209f1c98d13c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:18 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:35688
I20260929 19:05:18.827876 29223 raft_consensus.cc:2229] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:18.827973 29223 raft_consensus.cc:2258] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:18.828289 28669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 19:05:18.828437 29223 ts_tablet_manager.cc:1905] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:05:18.828449 28669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:18.829938 28947 tablet_service.cc:1467] Processing CreateTablet for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 (DEFAULT_TABLE table=test-table [id=49ac1bee94fc4b3aad4da60bf37ac14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:18.830056 28947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.830204 29087 tablet_service.cc:1467] Processing CreateTablet for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 (DEFAULT_TABLE table=test-table [id=49ac1bee94fc4b3aad4da60bf37ac14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:18.830344 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.831372 29177 tablet_bootstrap.cc:492] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: Bootstrap starting.
I20260929 19:05:18.831700 29176 tablet_bootstrap.cc:492] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: Bootstrap starting.
I20260929 19:05:18.831817 29177 tablet_bootstrap.cc:654] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.832070 29176 tablet_bootstrap.cc:654] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.832510 28807 tablet_service.cc:1467] Processing CreateTablet for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 (DEFAULT_TABLE table=test-table [id=49ac1bee94fc4b3aad4da60bf37ac14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:18.832635 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:18.832636 29225 tablet_replica.cc:330] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: stopping tablet replica
I20260929 19:05:18.832752 29225 raft_consensus.cc:2229] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:18.832804 29225 raft_consensus.cc:2258] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:18.834093 29178 tablet_bootstrap.cc:492] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: Bootstrap starting.
I20260929 19:05:18.834568 29225 ts_tablet_manager.cc:1905] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:18.834684 29178 tablet_bootstrap.cc:654] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:18.835974 29223 ts_tablet_manager.cc:1918] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:18.836110 29223 log.cc:1196] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/8657358fe2854ca0a09209f1c98d13c8
I20260929 19:05:18.836454 29225 ts_tablet_manager.cc:1918] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:18.836479 29223 ts_tablet_manager.cc:1939] T 8657358fe2854ca0a09209f1c98d13c8 P b891b380fa19460ead53c4b862fa1cd1: Deleting consensus metadata
I20260929 19:05:18.836623 29225 log.cc:1196] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/8657358fe2854ca0a09209f1c98d13c8
I20260929 19:05:18.836683 29176 tablet_bootstrap.cc:492] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: No bootstrap required, opened a new log
I20260929 19:05:18.836735 29176 ts_tablet_manager.cc:1397] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260929 19:05:18.836911 28656 catalog_manager.cc:4841] TS b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487): tablet 8657358fe2854ca0a09209f1c98d13c8 (table test-table [id=5ba8370306f74f9da93690fafc52525b]) successfully deleted
I20260929 19:05:18.836938 29225 ts_tablet_manager.cc:1939] T 8657358fe2854ca0a09209f1c98d13c8 P 78fb1171e86c4f16928230d1cc43fdfe: Deleting consensus metadata
I20260929 19:05:18.836902 29176 raft_consensus.cc:348] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.836977 29176 raft_consensus.cc:374] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.837002 29176 raft_consensus.cc:729] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb1171e86c4f16928230d1cc43fdfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.837051 29176 consensus_queue.cc:260] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.837194 29176 ts_tablet_manager.cc:1428] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.837534 29177 tablet_bootstrap.cc:492] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: No bootstrap required, opened a new log
I20260929 19:05:18.837579 29177 ts_tablet_manager.cc:1397] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260929 19:05:18.837715 29177 raft_consensus.cc:348] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.837777 29177 raft_consensus.cc:374] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.837798 29177 raft_consensus.cc:729] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b891b380fa19460ead53c4b862fa1cd1, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.837860 29177 consensus_queue.cc:260] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.837953 29177 ts_tablet_manager.cc:1428] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.838109 28654 catalog_manager.cc:4841] TS 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255): tablet 8657358fe2854ca0a09209f1c98d13c8 (table test-table [id=5ba8370306f74f9da93690fafc52525b]) successfully deleted
I20260929 19:05:18.838258 29178 tablet_bootstrap.cc:492] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: No bootstrap required, opened a new log
I20260929 19:05:18.838322 29178 ts_tablet_manager.cc:1397] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260929 19:05:18.838476 29178 raft_consensus.cc:348] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.838537 29178 raft_consensus.cc:374] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:18.838557 29178 raft_consensus.cc:729] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a272539b0f54f8d89f26435eb66ccfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:18.838603 29178 consensus_queue.cc:260] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.838686 29178 ts_tablet_manager.cc:1428] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:18.839320 29224 tablet_replica.cc:330] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: stopping tablet replica
I20260929 19:05:18.839519 29224 raft_consensus.cc:2229] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:18.839589 29224 raft_consensus.cc:2258] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:18.840031 29224 ts_tablet_manager.cc:1905] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:18.841627 29224 ts_tablet_manager.cc:1918] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:18.841701 29224 log.cc:1196] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/8657358fe2854ca0a09209f1c98d13c8
I20260929 19:05:18.842043 29224 ts_tablet_manager.cc:1939] T 8657358fe2854ca0a09209f1c98d13c8 P 5a272539b0f54f8d89f26435eb66ccfe: Deleting consensus metadata
I20260929 19:05:18.842471 28657 catalog_manager.cc:4841] TS 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181): tablet 8657358fe2854ca0a09209f1c98d13c8 (table test-table [id=5ba8370306f74f9da93690fafc52525b]) successfully deleted
I20260929 19:05:18.991752 29218 raft_consensus.cc:482] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:18.991858 29218 raft_consensus.cc:504] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.992017 29218 leader_election.cc:288] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181), 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255)
I20260929 19:05:18.992265 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfdf84bc2a3464fa3e1fb5ebf089225" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe" is_pre_election: true
I20260929 19:05:18.992353 28827 raft_consensus.cc:2454] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 0.
I20260929 19:05:18.992271 29107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfdf84bc2a3464fa3e1fb5ebf089225" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb1171e86c4f16928230d1cc43fdfe" is_pre_election: true
I20260929 19:05:18.992396 29107 raft_consensus.cc:2454] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 0.
I20260929 19:05:18.992527 28901 leader_election.cc:302] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a272539b0f54f8d89f26435eb66ccfe, b891b380fa19460ead53c4b862fa1cd1; no voters:
I20260929 19:05:18.992672 29218 raft_consensus.cc:2781] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:18.992717 29218 raft_consensus.cc:482] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:18.992741 29218 raft_consensus.cc:3037] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.993425 29218 raft_consensus.cc:504] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.993566 29218 leader_election.cc:288] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181), 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255)
I20260929 19:05:18.993744 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfdf84bc2a3464fa3e1fb5ebf089225" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe"
I20260929 19:05:18.993862 28827 raft_consensus.cc:3037] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.994457 28827 raft_consensus.cc:2454] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 1.
I20260929 19:05:18.994581 29107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfdf84bc2a3464fa3e1fb5ebf089225" candidate_uuid: "b891b380fa19460ead53c4b862fa1cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb1171e86c4f16928230d1cc43fdfe"
I20260929 19:05:18.994629 28901 leader_election.cc:302] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a272539b0f54f8d89f26435eb66ccfe, b891b380fa19460ead53c4b862fa1cd1; no voters:
I20260929 19:05:18.994684 29107 raft_consensus.cc:3037] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:18.994853 29218 raft_consensus.cc:2781] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:18.994921 29218 raft_consensus.cc:686] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 1 LEADER]: Becoming Leader. State: Replica: b891b380fa19460ead53c4b862fa1cd1, State: Running, Role: LEADER
I20260929 19:05:18.994982 29218 consensus_queue.cc:237] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:18.995476 29107 raft_consensus.cc:2454] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b891b380fa19460ead53c4b862fa1cd1 in term 1.
I20260929 19:05:18.995635 28669 catalog_manager.cc:5495] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "b891b380fa19460ead53c4b862fa1cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } health_report { overall_health: UNKNOWN } } }
I20260929 19:05:19.019898 28827 raft_consensus.cc:1261] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Refusing update from remote peer b891b380fa19460ead53c4b862fa1cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:19.020025 29107 raft_consensus.cc:1261] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Refusing update from remote peer b891b380fa19460ead53c4b862fa1cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:19.020159 29184 consensus_queue.cc:1035] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:19.020287 29218 consensus_queue.cc:1035] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:19.021553 29207 mvcc.cc:205] Tried to move back new op lower bound from 7334742913104158720 to 7334742913003917312. Current Snapshot: MvccSnapshot[applied={T|T < 7334742913104158720}]
W20260929 19:05:19.024947 29205 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 19:05:20.544144 28669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:20.544245 28669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:20.544993 28669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Sending DeleteTablet for 3 replicas of tablet 6dfdf84bc2a3464fa3e1fb5ebf089225
I20260929 19:05:20.545245 28807 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:20 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:54824
I20260929 19:05:20.545254 29087 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:20 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:35688
I20260929 19:05:20.545249 28947 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:20 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:50010
I20260929 19:05:20.545473 29256 tablet_replica.cc:330] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: stopping tablet replica
I20260929 19:05:20.545527 29254 tablet_replica.cc:330] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: stopping tablet replica
I20260929 19:05:20.545565 29256 raft_consensus.cc:2229] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:20.545643 29254 raft_consensus.cc:2229] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:20.545655 29256 raft_consensus.cc:2258] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:20.545740 29254 raft_consensus.cc:2258] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:20.545934 29256 ts_tablet_manager.cc:1905] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:20.546104 29254 ts_tablet_manager.cc:1905] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:20.547689 29255 tablet_replica.cc:330] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: stopping tablet replica
I20260929 19:05:20.547824 29256 ts_tablet_manager.cc:1918] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:05:20.547889 29256 log.cc:1196] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/6dfdf84bc2a3464fa3e1fb5ebf089225
I20260929 19:05:20.547888 29254 ts_tablet_manager.cc:1918] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:05:20.547933 29254 log.cc:1196] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/6dfdf84bc2a3464fa3e1fb5ebf089225
I20260929 19:05:20.548120 29256 ts_tablet_manager.cc:1939] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P b891b380fa19460ead53c4b862fa1cd1: Deleting consensus metadata
I20260929 19:05:20.548170 29254 ts_tablet_manager.cc:1939] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 5a272539b0f54f8d89f26435eb66ccfe: Deleting consensus metadata
I20260929 19:05:20.548425 29255 raft_consensus.cc:2229] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:20.548491 29255 raft_consensus.cc:2258] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:20.548465 28657 catalog_manager.cc:4841] TS 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181): tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 (table test-table [id=49ac1bee94fc4b3aad4da60bf37ac14e]) successfully deleted
I20260929 19:05:20.548475 28656 catalog_manager.cc:4841] TS b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487): tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 (table test-table [id=49ac1bee94fc4b3aad4da60bf37ac14e]) successfully deleted
I20260929 19:05:20.548741 29255 ts_tablet_manager.cc:1905] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:20.550091 29255 ts_tablet_manager.cc:1918] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:05:20.550140 29255 log.cc:1196] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/6dfdf84bc2a3464fa3e1fb5ebf089225
I20260929 19:05:20.550350 29255 ts_tablet_manager.cc:1939] T 6dfdf84bc2a3464fa3e1fb5ebf089225 P 78fb1171e86c4f16928230d1cc43fdfe: Deleting consensus metadata
I20260929 19:05:20.550683 28654 catalog_manager.cc:4841] TS 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255): tablet 6dfdf84bc2a3464fa3e1fb5ebf089225 (table test-table [id=49ac1bee94fc4b3aad4da60bf37ac14e]) successfully deleted
Sep 29 19:05:20 dist-test-slave-xz51 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:05:20 dist-test-slave-xz51 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:20.570283 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.562910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.26.78.254:40375)
0929 19:05:20.563039 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:20.563046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:20.563106 (+ 60us) server_negotiation.cc:407] Connection header received
0929 19:05:20.563169 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:20.563169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:20.563179 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:20.563256 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:20.563839 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.564358 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.565097 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.565322 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.565400 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.567750 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:20.567752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:20.567752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:20.567760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:20.569063 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:20.569435 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:20.569436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:20.569436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:20.569501 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:20.569703 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:20.569704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:20.569704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:20.569879 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:20.569978 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:20.570082 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 19:05:20.570169 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260929 19:05:20.571729 28669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:20.571882 28669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:20.573230 28947 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac (DEFAULT_TABLE table=test-table [id=980e50930a4d42c4a796f2363344e3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:20.573381 28947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:20.573570 29087 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac (DEFAULT_TABLE table=test-table [id=980e50930a4d42c4a796f2363344e3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:20.573745 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:20.574924 28807 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac (DEFAULT_TABLE table=test-table [id=980e50930a4d42c4a796f2363344e3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:20.575038 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:20.575570 29265 tablet_bootstrap.cc:492] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: Bootstrap starting.
I20260929 19:05:20.575964 29265 tablet_bootstrap.cc:654] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:20.576043 29266 tablet_bootstrap.cc:492] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: Bootstrap starting.
I20260929 19:05:20.576440 29266 tablet_bootstrap.cc:654] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:20.576723 29265 tablet_bootstrap.cc:492] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: No bootstrap required, opened a new log
I20260929 19:05:20.576782 29265 ts_tablet_manager.cc:1397] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:20.576895 29265 raft_consensus.cc:348] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.576989 29265 raft_consensus.cc:374] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:20.577013 29265 raft_consensus.cc:729] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b891b380fa19460ead53c4b862fa1cd1, State: Initialized, Role: FOLLOWER
I20260929 19:05:20.577068 29265 consensus_queue.cc:260] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.577134 29266 tablet_bootstrap.cc:492] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: No bootstrap required, opened a new log
I20260929 19:05:20.577180 29265 ts_tablet_manager.cc:1428] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:20.577181 29266 ts_tablet_manager.cc:1397] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:20.577356 29268 tablet_bootstrap.cc:492] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: Bootstrap starting.
I20260929 19:05:20.577318 29266 raft_consensus.cc:348] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.577399 29266 raft_consensus.cc:374] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:20.577418 29266 raft_consensus.cc:729] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb1171e86c4f16928230d1cc43fdfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:20.577525 29266 consensus_queue.cc:260] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.577719 29266 ts_tablet_manager.cc:1428] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:20.577754 29268 tablet_bootstrap.cc:654] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:20.578722 29268 tablet_bootstrap.cc:492] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: No bootstrap required, opened a new log
I20260929 19:05:20.578789 29268 ts_tablet_manager.cc:1397] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:20.578896 29268 raft_consensus.cc:348] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.578981 29268 raft_consensus.cc:374] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:20.579012 29268 raft_consensus.cc:729] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a272539b0f54f8d89f26435eb66ccfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:20.579084 29268 consensus_queue.cc:260] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.579201 29268 ts_tablet_manager.cc:1428] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:20.647843 29270 raft_consensus.cc:482] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:20.647967 29270 raft_consensus.cc:504] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.648393 29270 leader_election.cc:288] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181), b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487)
I20260929 19:05:20.651633 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.648840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:35609 (local address 127.26.78.193:35181)
0929 19:05:20.649009 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:20.649011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:20.649022 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:05:20.649043 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:20.649044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:20.649052 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:20.649132 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:20.649280 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.649860 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.650878 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.651261 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.651359 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.651369 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:20.651508 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 19:05:20.651549 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260929 19:05:20.651666 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.648555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:35181 (local address 127.26.78.195:35609)
0929 19:05:20.648774 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:20.648786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:20.648840 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:20.649135 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:20.649135 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:20.649138 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:20.649195 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.649196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.649959 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.649960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.650763 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.650765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.651358 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.651359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.651426 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.651429 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:20.651501 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:05:20.651543 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":4,"thread_start_us":122,"threads_started":1}
I20260929 19:05:20.651696 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.648552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39487 (local address 127.26.78.195:59339)
0929 19:05:20.648770 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:20.648791 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:20.648882 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:20.649030 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:20.649031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:20.649033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:20.649106 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.649108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.649755 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.649756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.650761 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.650764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.651383 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.651384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.651466 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.651469 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:20.651546 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 19:05:20.651590 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":1,"thread_start_us":137,"threads_started":1}
I20260929 19:05:20.651679 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.648617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:59339 (local address 127.26.78.194:39487)
0929 19:05:20.648774 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:20.648777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:20.648873 (+ 96us) server_negotiation.cc:407] Connection header received
0929 19:05:20.648942 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:20.648943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:20.648955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:20.649023 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:20.649181 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.649667 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.650887 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.651278 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.651385 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.651394 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:20.651555 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 19:05:20.651599 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260929 19:05:20.651970 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2b72c9e29b4b9aa9b2d0ede6df4bac" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891b380fa19460ead53c4b862fa1cd1" is_pre_election: true
I20260929 19:05:20.652129 28967 raft_consensus.cc:2454] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 0.
I20260929 19:05:20.652114 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2b72c9e29b4b9aa9b2d0ede6df4bac" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe" is_pre_election: true
I20260929 19:05:20.652199 28827 raft_consensus.cc:2454] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 0.
I20260929 19:05:20.652360 29040 leader_election.cc:302] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb1171e86c4f16928230d1cc43fdfe, b891b380fa19460ead53c4b862fa1cd1; no voters:
I20260929 19:05:20.652510 29270 raft_consensus.cc:2781] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:20.652555 29270 raft_consensus.cc:482] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:20.652577 29270 raft_consensus.cc:3037] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:20.653242 29270 raft_consensus.cc:504] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.653388 29270 leader_election.cc:288] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 election: Requested vote from peers 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181), b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487)
I20260929 19:05:20.653532 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2b72c9e29b4b9aa9b2d0ede6df4bac" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe"
I20260929 19:05:20.653529 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2b72c9e29b4b9aa9b2d0ede6df4bac" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891b380fa19460ead53c4b862fa1cd1"
I20260929 19:05:20.653609 28967 raft_consensus.cc:3037] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:20.653631 28827 raft_consensus.cc:3037] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:20.654320 28967 raft_consensus.cc:2454] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 1.
I20260929 19:05:20.654345 28827 raft_consensus.cc:2454] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 1.
I20260929 19:05:20.654489 29040 leader_election.cc:302] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb1171e86c4f16928230d1cc43fdfe, b891b380fa19460ead53c4b862fa1cd1; no voters:
I20260929 19:05:20.654592 29270 raft_consensus.cc:2781] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:20.654736 29270 raft_consensus.cc:686] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 1 LEADER]: Becoming Leader. State: Replica: 78fb1171e86c4f16928230d1cc43fdfe, State: Running, Role: LEADER
I20260929 19:05:20.654814 29270 consensus_queue.cc:237] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.655504 28669 catalog_manager.cc:5495] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe reported cstate change: term changed from 0 to 1, leader changed from <none> to 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195). New cstate: current_term: 1 leader_uuid: "78fb1171e86c4f16928230d1cc43fdfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } health_report { overall_health: HEALTHY } } }
I20260929 19:05:20.673317 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.670942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.26.78.195:36255)
0929 19:05:20.671084 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:20.671087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:20.671095 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:20.671140 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:20.671141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:20.671159 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:20.671231 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:20.671384 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.671872 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.672590 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.672780 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.672830 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.672967 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:20.673071 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:20.673127 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:20.673161 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 19:05:20.673177 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
I20260929 19:05:20.677999 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.675709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:53461 (local address 127.26.78.193:35181)
0929 19:05:20.675769 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:20.675771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:20.675896 (+ 125us) server_negotiation.cc:407] Connection header received
0929 19:05:20.675936 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:20.675936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:20.675944 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:20.675990 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:20.676091 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.676623 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.677380 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.677556 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.677604 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.677752 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:20.677777 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:20.677860 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:20.677925 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 19:05:20.677943 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:05:20.678057 29282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.675648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:35181 (local address 127.26.78.195:53461)
0929 19:05:20.675825 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:20.675837 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:20.675895 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:20.676004 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:20.676004 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:20.676005 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:20.676055 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.676056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.676700 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.676701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.677307 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.677309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.677631 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.677632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.677675 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.677686 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:20.677873 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:20.677874 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:05:20.677923 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:05:20.677956 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260929 19:05:20.680109 28827 raft_consensus.cc:1261] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Refusing update from remote peer 78fb1171e86c4f16928230d1cc43fdfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:20.680160 28967 raft_consensus.cc:1261] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Refusing update from remote peer 78fb1171e86c4f16928230d1cc43fdfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:20.680382 29276 consensus_queue.cc:1035] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:20.680509 29270 consensus_queue.cc:1035] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:20.687906 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.685542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:49537 (local address 127.26.78.195:36255)
0929 19:05:20.685605 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:20.685607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:20.685716 (+ 109us) server_negotiation.cc:407] Connection header received
0929 19:05:20.685764 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:20.685764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:20.685798 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:20.685900 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:20.686013 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.686467 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.687236 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.687448 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.687520 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.687670 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:20.687702 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:20.687753 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:20.687815 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 19:05:20.687837 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:05:20.687933 29287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:20.685510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:36255 (local address 127.26.78.193:49537)
0929 19:05:20.685646 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:20.685657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:20.685718 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:20.685905 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:20.685906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:20.685908 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:05:20.685961 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.685963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.686530 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.686531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.687138 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:20.687140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:20.687518 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:20.687519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:20.687575 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:20.687579 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:05:20.687761 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:05:20.687763 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:20.687815 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:05:20.687856 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260929 19:05:20.693986 28668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:20.694092 28668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:20.694684 28668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Sending DeleteTablet for 3 replicas of tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac
I20260929 19:05:20.694932 28807 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:20 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:54824
I20260929 19:05:20.694955 28947 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:20 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:50010
I20260929 19:05:20.695034 29254 tablet_replica.cc:330] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: stopping tablet replica
I20260929 19:05:20.695050 29256 tablet_replica.cc:330] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: stopping tablet replica
I20260929 19:05:20.695101 29254 raft_consensus.cc:2229] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:20.695104 29256 raft_consensus.cc:2229] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:20.695137 29256 raft_consensus.cc:2258] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:20.695140 29254 raft_consensus.cc:2258] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:20.695286 29087 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:20 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:35688
I20260929 19:05:20.695379 29255 tablet_replica.cc:330] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: stopping tablet replica
I20260929 19:05:20.695447 29255 raft_consensus.cc:2229] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:20.695525 29255 raft_consensus.cc:2258] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:20.695636 29256 ts_tablet_manager.cc:1905] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:20.695689 28668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 19:05:20.695798 29254 ts_tablet_manager.cc:1905] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:05:20.695829 28668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:20.697371 28947 tablet_service.cc:1467] Processing CreateTablet for tablet bcc88275ebe244f5849f0c5434cb4e1f (DEFAULT_TABLE table=test-table [id=97070a2c1d1441c8b31c46988c52021a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:20.697516 29087 tablet_service.cc:1467] Processing CreateTablet for tablet bcc88275ebe244f5849f0c5434cb4e1f (DEFAULT_TABLE table=test-table [id=97070a2c1d1441c8b31c46988c52021a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:20.697525 29255 ts_tablet_manager.cc:1905] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:20.697576 28807 tablet_service.cc:1467] Processing CreateTablet for tablet bcc88275ebe244f5849f0c5434cb4e1f (DEFAULT_TABLE table=test-table [id=97070a2c1d1441c8b31c46988c52021a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:20.697649 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc88275ebe244f5849f0c5434cb4e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:20.698091 29254 ts_tablet_manager.cc:1918] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:20.698148 29254 log.cc:1196] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/4a2b72c9e29b4b9aa9b2d0ede6df4bac
I20260929 19:05:20.698376 29254 ts_tablet_manager.cc:1939] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 5a272539b0f54f8d89f26435eb66ccfe: Deleting consensus metadata
I20260929 19:05:20.698504 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc88275ebe244f5849f0c5434cb4e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:20.698930 29266 tablet_bootstrap.cc:492] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: Bootstrap starting.
I20260929 19:05:20.698973 29255 ts_tablet_manager.cc:1918] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:20.699000 29256 ts_tablet_manager.cc:1918] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:05:20.699023 29255 log.cc:1196] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/4a2b72c9e29b4b9aa9b2d0ede6df4bac
I20260929 19:05:20.699046 29256 log.cc:1196] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/4a2b72c9e29b4b9aa9b2d0ede6df4bac
I20260929 19:05:20.699244 29255 ts_tablet_manager.cc:1939] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P 78fb1171e86c4f16928230d1cc43fdfe: Deleting consensus metadata
I20260929 19:05:20.699250 29256 ts_tablet_manager.cc:1939] T 4a2b72c9e29b4b9aa9b2d0ede6df4bac P b891b380fa19460ead53c4b862fa1cd1: Deleting consensus metadata
I20260929 19:05:20.699337 29266 tablet_bootstrap.cc:654] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:20.699378 28947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc88275ebe244f5849f0c5434cb4e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:20.699636 28654 catalog_manager.cc:4841] TS 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255): tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac (table test-table [id=980e50930a4d42c4a796f2363344e3c1]) successfully deleted
I20260929 19:05:20.699800 29268 tablet_bootstrap.cc:492] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe: Bootstrap starting.
I20260929 19:05:20.700124 29268 tablet_bootstrap.cc:654] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:20.700351 28657 catalog_manager.cc:4841] TS 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181): tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac (table test-table [id=980e50930a4d42c4a796f2363344e3c1]) successfully deleted
I20260929 19:05:20.700507 29265 tablet_bootstrap.cc:492] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: Bootstrap starting.
I20260929 19:05:20.700685 28656 catalog_manager.cc:4841] TS b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487): tablet 4a2b72c9e29b4b9aa9b2d0ede6df4bac (table test-table [id=980e50930a4d42c4a796f2363344e3c1]) successfully deleted
I20260929 19:05:20.700776 29265 tablet_bootstrap.cc:654] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:20.701293 29266 tablet_bootstrap.cc:492] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: No bootstrap required, opened a new log
I20260929 19:05:20.701334 29265 tablet_bootstrap.cc:492] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: No bootstrap required, opened a new log
I20260929 19:05:20.701363 29266 ts_tablet_manager.cc:1397] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:20.701376 29265 ts_tablet_manager.cc:1397] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 19:05:20.701475 29265 raft_consensus.cc:348] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.701534 29265 raft_consensus.cc:374] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:20.701553 29265 raft_consensus.cc:729] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b891b380fa19460ead53c4b862fa1cd1, State: Initialized, Role: FOLLOWER
I20260929 19:05:20.701550 29266 raft_consensus.cc:348] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.701601 29265 consensus_queue.cc:260] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.701635 29266 raft_consensus.cc:374] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:20.701658 29266 raft_consensus.cc:729] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb1171e86c4f16928230d1cc43fdfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:20.701687 29265 ts_tablet_manager.cc:1428] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:20.701701 29266 consensus_queue.cc:260] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.701867 29266 ts_tablet_manager.cc:1428] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:20.706213 29268 tablet_bootstrap.cc:492] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe: No bootstrap required, opened a new log
I20260929 19:05:20.706293 29268 ts_tablet_manager.cc:1397] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260929 19:05:20.706426 29268 raft_consensus.cc:348] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.706506 29268 raft_consensus.cc:374] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:20.706538 29268 raft_consensus.cc:729] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a272539b0f54f8d89f26435eb66ccfe, State: Initialized, Role: FOLLOWER
I20260929 19:05:20.706588 29268 consensus_queue.cc:260] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.706702 29268 ts_tablet_manager.cc:1428] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:20.720975 29270 raft_consensus.cc:482] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:20.721086 29270 raft_consensus.cc:504] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.721242 29270 leader_election.cc:288] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181), b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487)
I20260929 19:05:20.721434 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc88275ebe244f5849f0c5434cb4e1f" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe" is_pre_election: true
I20260929 19:05:20.721442 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc88275ebe244f5849f0c5434cb4e1f" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891b380fa19460ead53c4b862fa1cd1" is_pre_election: true
I20260929 19:05:20.721535 28967 raft_consensus.cc:2454] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 0.
I20260929 19:05:20.721557 28827 raft_consensus.cc:2454] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 0.
I20260929 19:05:20.721691 29040 leader_election.cc:302] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb1171e86c4f16928230d1cc43fdfe, b891b380fa19460ead53c4b862fa1cd1; no voters:
I20260929 19:05:20.721813 29270 raft_consensus.cc:2781] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:20.721867 29270 raft_consensus.cc:482] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:20.721887 29270 raft_consensus.cc:3037] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:20.722707 29270 raft_consensus.cc:504] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.722846 29270 leader_election.cc:288] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 election: Requested vote from peers 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181), b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487)
I20260929 19:05:20.723027 28827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc88275ebe244f5849f0c5434cb4e1f" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a272539b0f54f8d89f26435eb66ccfe"
I20260929 19:05:20.723109 28827 raft_consensus.cc:3037] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:20.723101 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc88275ebe244f5849f0c5434cb4e1f" candidate_uuid: "78fb1171e86c4f16928230d1cc43fdfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891b380fa19460ead53c4b862fa1cd1"
I20260929 19:05:20.723169 28967 raft_consensus.cc:3037] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:20.723671 28827 raft_consensus.cc:2454] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 1.
I20260929 19:05:20.723865 29041 leader_election.cc:302] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a272539b0f54f8d89f26435eb66ccfe, 78fb1171e86c4f16928230d1cc43fdfe; no voters:
I20260929 19:05:20.723940 28967 raft_consensus.cc:2454] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fb1171e86c4f16928230d1cc43fdfe in term 1.
I20260929 19:05:20.723975 29270 raft_consensus.cc:2781] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:20.724014 29270 raft_consensus.cc:686] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 1 LEADER]: Becoming Leader. State: Replica: 78fb1171e86c4f16928230d1cc43fdfe, State: Running, Role: LEADER
I20260929 19:05:20.724062 29270 consensus_queue.cc:237] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } }
I20260929 19:05:20.724658 28668 catalog_manager.cc:5495] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe reported cstate change: term changed from 0 to 1, leader changed from <none> to 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195). New cstate: current_term: 1 leader_uuid: "78fb1171e86c4f16928230d1cc43fdfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78fb1171e86c4f16928230d1cc43fdfe" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36255 } health_report { overall_health: HEALTHY } } }
I20260929 19:05:20.731217 28827 raft_consensus.cc:1261] T bcc88275ebe244f5849f0c5434cb4e1f P 5a272539b0f54f8d89f26435eb66ccfe [term 1 FOLLOWER]: Refusing update from remote peer 78fb1171e86c4f16928230d1cc43fdfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:20.731480 28967 raft_consensus.cc:1261] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Refusing update from remote peer 78fb1171e86c4f16928230d1cc43fdfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:20.731478 29285 consensus_queue.cc:1035] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a272539b0f54f8d89f26435eb66ccfe" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 35181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:20.731698 29285 consensus_queue.cc:1035] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b891b380fa19460ead53c4b862fa1cd1" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:20.732707 29233 mvcc.cc:205] Tried to move back new op lower bound from 7334742920113922048 to 7334742920086196224. Current Snapshot: MvccSnapshot[applied={T|T < 7334742920113922048}]
W20260929 19:05:20.736611 29236 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 19:05:22.256258 28669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:22.256373 28669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:22.257261 28669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afce3a3a682f45c4a0a25029637ff3d6: Sending DeleteTablet for 3 replicas of tablet bcc88275ebe244f5849f0c5434cb4e1f
I20260929 19:05:22.257517 28947 tablet_service.cc:1514] Processing DeleteTablet for tablet bcc88275ebe244f5849f0c5434cb4e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:22 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:50010
I20260929 19:05:22.257553 29087 tablet_service.cc:1514] Processing DeleteTablet for tablet bcc88275ebe244f5849f0c5434cb4e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:22 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:35688
I20260929 19:05:22.257571 28807 tablet_service.cc:1514] Processing DeleteTablet for tablet bcc88275ebe244f5849f0c5434cb4e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:22 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:54824
I20260929 19:05:22.257720 29317 tablet_replica.cc:330] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: stopping tablet replica
I20260929 19:05:22.257753 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 28742
I20260929 19:05:22.257836 29317 raft_consensus.cc:2229] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:22.259447 29317 raft_consensus.cc:2258] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:22.259586 29318 tablet_replica.cc:330] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: stopping tablet replica
I20260929 19:05:22.259648 29318 raft_consensus.cc:2229] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:22.259712 29318 raft_consensus.cc:2258] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:22.259917 29317 ts_tablet_manager.cc:1905] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:22.260004 29318 ts_tablet_manager.cc:1905] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:22.261701 29317 ts_tablet_manager.cc:1918] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:05:22.261762 29317 log.cc:1196] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/bcc88275ebe244f5849f0c5434cb4e1f
I20260929 19:05:22.262045 29318 ts_tablet_manager.cc:1918] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:05:22.262065 29317 ts_tablet_manager.cc:1939] T bcc88275ebe244f5849f0c5434cb4e1f P b891b380fa19460ead53c4b862fa1cd1: Deleting consensus metadata
I20260929 19:05:22.262121 29318 log.cc:1196] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/bcc88275ebe244f5849f0c5434cb4e1f
I20260929 19:05:22.262372 29318 ts_tablet_manager.cc:1939] T bcc88275ebe244f5849f0c5434cb4e1f P 78fb1171e86c4f16928230d1cc43fdfe: Deleting consensus metadata
I20260929 19:05:22.262415 28656 catalog_manager.cc:4841] TS b891b380fa19460ead53c4b862fa1cd1 (127.26.78.194:39487): tablet bcc88275ebe244f5849f0c5434cb4e1f (table test-table [id=97070a2c1d1441c8b31c46988c52021a]) successfully deleted
I20260929 19:05:22.263141 28654 catalog_manager.cc:4841] TS 78fb1171e86c4f16928230d1cc43fdfe (127.26.78.195:36255): tablet bcc88275ebe244f5849f0c5434cb4e1f (table test-table [id=97070a2c1d1441c8b31c46988c52021a]) successfully deleted
W20260929 19:05:22.265574 28657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:35181 (error 108)
I20260929 19:05:22.265928 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 28882
I20260929 19:05:22.270005 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:22.266022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:35181 (local address 127.0.0.1:54842)
0929 19:05:22.269917 (+ 3895us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:22.269967 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3861,"thread_start_us":61,"threads_started":1}
W20260929 19:05:22.270156 28657 catalog_manager.cc:4584] TS 5a272539b0f54f8d89f26435eb66ccfe (127.26.78.193:35181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bcc88275ebe244f5849f0c5434cb4e1f: Network error: Client connection negotiation failed: client connection to 127.26.78.193:35181: connect: Connection refused (error 111)
I20260929 19:05:22.272092 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 29022
I20260929 19:05:22.277623 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 28638
2026-09-29T19:05:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5657 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:22 dist-test-slave-xz51 krb5kdc[29324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:22 dist-test-slave-xz51 krb5kdc[29324](info): set up 2 sockets
Sep 29 19:05:22 dist-test-slave-xz51 krb5kdc[29324](info): commencing operation
krb5kdc: starting...
W20260929 19:05:23.317772 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:23 dist-test-slave-xz51 krb5kdc[29324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:23.436371 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43621
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:43621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:23.516744 29337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:23.517032 29337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:23.517069 29337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:23.518242 29337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:23.518314 29337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:23.518339 29337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:23.518360 29337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:23.518386 29337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:23.519877 29337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:43621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:23.520148 29337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:23.520380 29337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:05:23.525336 29337 server_base.cc:1034] running on GCE node
W20260929 19:05:23.525269 29345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:23.525276 29343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:23.525276 29342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:23.525805 29337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:23.526160 29337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:23.527329 29337 hybrid_clock.cc:648] HybridClock initialized: now 1790708723527311 us; error 38 us; skew 500 ppm
Sep 29 19:05:23 dist-test-slave-xz51 krb5kdc[29324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708723, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:23.534940 29337 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:23.535362 29337 webserver.cc:460] Webserver started at http://127.26.78.254:44255/ using document root <none> and password file <none>
I20260929 19:05:23.535580 29337 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:23.535624 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:23.535714 29337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:23.536751 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "e9b48594c3ff4407b52251b83d641560"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "02692be0a119e51d9671f5e4d2330641"
server_key_iv: "c62ba5a0f5cb4bfab360cde19ac9c973"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.537048 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "e9b48594c3ff4407b52251b83d641560"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "02692be0a119e51d9671f5e4d2330641"
server_key_iv: "c62ba5a0f5cb4bfab360cde19ac9c973"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.538414 29337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:23.539113 29352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:23.539273 29337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:23.539379 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "e9b48594c3ff4407b52251b83d641560"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "02692be0a119e51d9671f5e4d2330641"
server_key_iv: "c62ba5a0f5cb4bfab360cde19ac9c973"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.539451 29337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:23.553925 29337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:23.556279 29337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:23.556453 29337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:23.560616 29337 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:43621
I20260929 19:05:23.560652 29404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:43621 every 8 connection(s)
I20260929 19:05:23.561017 29337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:23.561928 29405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:23.564132 29405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560: Bootstrap starting.
I20260929 19:05:23.564648 29405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:23.564890 29405 log.cc:824] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:23.565590 29405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560: No bootstrap required, opened a new log
I20260929 19:05:23.567224 29405 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b48594c3ff4407b52251b83d641560" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43621 } }
I20260929 19:05:23.567368 29405 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:23.567390 29405 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b48594c3ff4407b52251b83d641560, State: Initialized, Role: FOLLOWER
I20260929 19:05:23.567540 29405 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b48594c3ff4407b52251b83d641560" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43621 } }
I20260929 19:05:23.567625 29405 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:23.567648 29405 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:23.567677 29405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:23.568161 29405 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b48594c3ff4407b52251b83d641560" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43621 } }
I20260929 19:05:23.568243 29405 leader_election.cc:302] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9b48594c3ff4407b52251b83d641560; no voters:
I20260929 19:05:23.568362 29405 leader_election.cc:288] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:23.568421 29408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:23.568580 29408 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [term 1 LEADER]: Becoming Leader. State: Replica: e9b48594c3ff4407b52251b83d641560, State: Running, Role: LEADER
I20260929 19:05:23.568615 29405 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:23.568727 29408 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b48594c3ff4407b52251b83d641560" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43621 } }
I20260929 19:05:23.569105 29410 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9b48594c3ff4407b52251b83d641560. Latest consensus state: current_term: 1 leader_uuid: "e9b48594c3ff4407b52251b83d641560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b48594c3ff4407b52251b83d641560" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43621 } } }
I20260929 19:05:23.569235 29409 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9b48594c3ff4407b52251b83d641560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b48594c3ff4407b52251b83d641560" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43621 } } }
I20260929 19:05:23.569263 29410 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:23.569324 29409 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:23.569703 29417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:23.570209 29417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:23.571108 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 29337
I20260929 19:05:23.571198 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:23.571398 26939 external_mini_cluster.cc:1428] Setting key 284301ca8b33cf37bc5bdfcef8192c6b
I20260929 19:05:23.571700 29417 catalog_manager.cc:1347] Generated new cluster ID: c68cbd37ca434c829b0df3f98fa26fba
I20260929 19:05:23.571750 29417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:23.580804 29417 catalog_manager.cc:1370] Generated new certificate authority record
Sep 29 19:05:23 dist-test-slave-xz51 krb5kdc[29324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:23.581540 29417 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:23.585381 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:23.573964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.26.78.254:43621)
0929 19:05:23.574119 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:23.574129 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:23.574164 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:05:23.574281 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:23.574288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:23.574342 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:23.574475 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:23.575274 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.576650 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.577770 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.578455 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.578548 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:23.581401 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:23.581406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:23.581406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:23.581418 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:23.583399 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:23.583808 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:23.583810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:23.583811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:23.583912 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:23.584119 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:23.584120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:23.584120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:23.584368 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:23.584512 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:23.584993 (+ 481us) server_negotiation.cc:299] Negotiation successful
0929 19:05:23.585146 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260929 19:05:23.591764 29417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9b48594c3ff4407b52251b83d641560: Generated new TSK 0
I20260929 19:05:23.591951 29417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:23.621461 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43621
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:23.702133 29431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:23.702374 29431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:23.702411 29431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:23.703560 29431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:23.703624 29431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:23.703666 29431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:23.705104 29431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:23.705374 29431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:23.705600 29431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:23.710465 29439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:23.710528 29431 server_base.cc:1034] running on GCE node
W20260929 19:05:23.710444 29437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:23.710444 29436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:23.710947 29431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:23.711210 29431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:23.712363 29431 hybrid_clock.cc:648] HybridClock initialized: now 1790708723712356 us; error 42 us; skew 500 ppm
Sep 29 19:05:23 dist-test-slave-xz51 krb5kdc[29324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708723, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:23.714819 29431 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:23.715273 29431 webserver.cc:460] Webserver started at http://127.26.78.193:44197/ using document root <none> and password file <none>
I20260929 19:05:23.715500 29431 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:23.715564 29431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:23.715705 29431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:23.716738 29431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "674d736389c9446081ba9cca74153519"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "f6cd8226440c67610a89e9a71273772f"
server_key_iv: "bcdf258fe454c76d91707f42db0647aa"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.717088 29431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "674d736389c9446081ba9cca74153519"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "f6cd8226440c67610a89e9a71273772f"
server_key_iv: "bcdf258fe454c76d91707f42db0647aa"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.718444 29431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:23.719137 29446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:23.719301 29431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:23.719394 29431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "674d736389c9446081ba9cca74153519"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "f6cd8226440c67610a89e9a71273772f"
server_key_iv: "bcdf258fe454c76d91707f42db0647aa"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.719470 29431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:23.748399 29431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:23.750326 29431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:23.750494 29431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:23.750727 29431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:23.751039 29431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:23.751092 29431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:23.751133 29431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:23.751173 29431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:23.756932 29431 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:35415
I20260929 19:05:23.756985 29559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:35415 every 8 connection(s)
I20260929 19:05:23.757311 29431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:05:23 dist-test-slave-xz51 krb5kdc[29324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708723, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:23.766503 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:23.758057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:58695 (local address 127.26.78.254:43621)
0929 19:05:23.758107 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:23.758109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:23.758289 (+ 180us) server_negotiation.cc:407] Connection header received
0929 19:05:23.758814 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:23.758815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:23.758828 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:23.758880 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:23.759925 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.760437 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.761450 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.761675 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.761763 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:23.763876 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:23.763877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:23.763878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:23.763885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:23.765081 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:23.765479 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:23.765480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:23.765481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:23.765542 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:23.765775 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:23.765776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:23.765777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:23.765975 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:23.766053 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:23.766316 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 19:05:23.766408 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:05:23.766692 29562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:23.758015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43621 (local address 127.26.78.193:58695)
0929 19:05:23.758163 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:23.758207 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:23.758706 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:23.758963 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:23.758971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:23.759596 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:23.759822 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:23.759827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.760548 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.760549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:23.761353 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:23.761355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.761776 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.761777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:23.761914 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:23.762480 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:23.762488 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:23.763779 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:23.765173 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:23.765180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:23.765182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:23.765405 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:23.765609 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:23.765610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:23.765610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:23.765712 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:23.766040 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:23.766047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:23.766183 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 19:05:23.766300 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 19:05:23.766460 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260929 19:05:23.766914 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 29431
I20260929 19:05:23.766994 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:23.767143 29560 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43621
I20260929 19:05:23.767181 26939 external_mini_cluster.cc:1428] Setting key dce7a80c6e264d4b20a3c38d38595d05
I20260929 19:05:23.767241 29560 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:23.767465 29560 heartbeater.cc:511] Master 127.26.78.254:43621 requested a full tablet report, sending...
I20260929 19:05:23.767967 29369 ts_manager.cc:194] Registered new tserver with Master: 674d736389c9446081ba9cca74153519 (127.26.78.193:35415)
I20260929 19:05:23.768617 29369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:58695
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:23.791942 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43621
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:23.877998 29565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:23.878259 29565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:23.878300 29565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:23.879565 29565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:23.879634 29565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:23.879688 29565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:23.881129 29565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:23.881397 29565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:23.881623 29565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:23.886546 29570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:23.886555 29571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:23.886565 29573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:23.886994 29565 server_base.cc:1034] running on GCE node
I20260929 19:05:23.887233 29565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:23.887486 29565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:23.888639 29565 hybrid_clock.cc:648] HybridClock initialized: now 1790708723888600 us; error 51 us; skew 500 ppm
Sep 29 19:05:23 dist-test-slave-xz51 krb5kdc[29324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708723, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:23.891245 29565 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:23.891633 29565 webserver.cc:460] Webserver started at http://127.26.78.194:42745/ using document root <none> and password file <none>
I20260929 19:05:23.891834 29565 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:23.891875 29565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:23.891961 29565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:23.892943 29565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "2ae7866fe6ef43a3b453e14914c8f0d9"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "feb9b07627e6902b0cfa0f6d6dc76f30"
server_key_iv: "aa95d60a7af106667ee15765f09d1888"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.893229 29565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "2ae7866fe6ef43a3b453e14914c8f0d9"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "feb9b07627e6902b0cfa0f6d6dc76f30"
server_key_iv: "aa95d60a7af106667ee15765f09d1888"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.894531 29565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:23.895298 29580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:23.895498 29565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:23.895596 29565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "2ae7866fe6ef43a3b453e14914c8f0d9"
format_stamp: "Formatted at 2026-09-29 19:05:23 on dist-test-slave-xz51"
server_key: "feb9b07627e6902b0cfa0f6d6dc76f30"
server_key_iv: "aa95d60a7af106667ee15765f09d1888"
server_key_version: "encryptionkey@0"
I20260929 19:05:23.895668 29565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:23.908179 29565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:23.910473 29565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:23.910619 29565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:23.910866 29565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:23.911178 29565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:23.911213 29565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:23.911253 29565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:23.911280 29565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:23.916811 29565 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:35703
I20260929 19:05:23.916870 29693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:35703 every 8 connection(s)
I20260929 19:05:23.917232 29565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:05:23 dist-test-slave-xz51 krb5kdc[29324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708723, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:23.926127 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:23.918021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:59989 (local address 127.26.78.254:43621)
0929 19:05:23.918079 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:23.918081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:23.918293 (+ 212us) server_negotiation.cc:407] Connection header received
0929 19:05:23.918759 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:23.918760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:23.918770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:23.918840 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:23.919737 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.920222 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.921181 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.921376 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.921426 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:23.923707 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:23.923708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:23.923708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:23.923714 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:23.924873 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:23.925255 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:23.925255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:23.925256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:23.925303 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:23.925516 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:23.925517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:23.925517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:23.925616 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:23.925666 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:23.925946 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 19:05:23.926034 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:05:23.926242 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:23.918026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43621 (local address 127.26.78.194:59989)
0929 19:05:23.918172 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:23.918209 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:23.918678 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:23.918894 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:23.918897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:23.919467 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:23.919660 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:23.919665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.920297 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.920297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:23.921110 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:23.921112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:23.921445 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:23.921446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:23.921550 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:23.922186 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:23.922197 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:23.923620 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:23.924942 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:23.924947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:23.924949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:23.925167 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:23.925363 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:23.925364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:23.925364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:23.925465 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:23.925666 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:23.925670 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:23.925782 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 19:05:23.925921 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 19:05:23.926046 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260929 19:05:23.926827 29694 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43621
I20260929 19:05:23.926931 29694 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:23.927107 29694 heartbeater.cc:511] Master 127.26.78.254:43621 requested a full tablet report, sending...
I20260929 19:05:23.927106 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 29565
I20260929 19:05:23.927196 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:23.927417 26939 external_mini_cluster.cc:1428] Setting key d4939a5c0dccba0126d0254747ed451a
I20260929 19:05:23.927536 29369 ts_manager.cc:194] Registered new tserver with Master: 2ae7866fe6ef43a3b453e14914c8f0d9 (127.26.78.194:35703)
I20260929 19:05:23.928023 29369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:59989
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:23.952867 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43621
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:24.033761 29699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:24.034092 29699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:24.034133 29699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:24.035329 29699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:24.035408 29699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:24.035451 29699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:24.036934 29699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:24.037227 29699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:24.037477 29699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:24.042611 29705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:24.042630 29704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:24.042629 29707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:24.042968 29699 server_base.cc:1034] running on GCE node
I20260929 19:05:24.043179 29699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:24.043382 29699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:24.044546 29699 hybrid_clock.cc:648] HybridClock initialized: now 1790708724044531 us; error 28 us; skew 500 ppm
Sep 29 19:05:24 dist-test-slave-xz51 krb5kdc[29324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708724, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:24.047151 29699 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:24.047607 29699 webserver.cc:460] Webserver started at http://127.26.78.195:35435/ using document root <none> and password file <none>
I20260929 19:05:24.047859 29699 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:24.047926 29699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:24.048066 29699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:24.049126 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "84e26f8275054978a548537a53aeb067"
format_stamp: "Formatted at 2026-09-29 19:05:24 on dist-test-slave-xz51"
server_key: "7c03b6e1509da14814666d25c4e8c61b"
server_key_iv: "4a898c7313ab0bf9f4b1e6aa017d62d5"
server_key_version: "encryptionkey@0"
I20260929 19:05:24.049499 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "84e26f8275054978a548537a53aeb067"
format_stamp: "Formatted at 2026-09-29 19:05:24 on dist-test-slave-xz51"
server_key: "7c03b6e1509da14814666d25c4e8c61b"
server_key_iv: "4a898c7313ab0bf9f4b1e6aa017d62d5"
server_key_version: "encryptionkey@0"
I20260929 19:05:24.050987 29699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:24.051851 29714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:24.052037 29699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:05:24.052139 29699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "84e26f8275054978a548537a53aeb067"
format_stamp: "Formatted at 2026-09-29 19:05:24 on dist-test-slave-xz51"
server_key: "7c03b6e1509da14814666d25c4e8c61b"
server_key_iv: "4a898c7313ab0bf9f4b1e6aa017d62d5"
server_key_version: "encryptionkey@0"
I20260929 19:05:24.052214 29699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:24.071209 29699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:24.073556 29699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:24.073725 29699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:24.074025 29699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:24.074334 29699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:24.074385 29699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:24.074425 29699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:24.074457 29699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:24.080083 29699 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:33721
I20260929 19:05:24.080173 29827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:33721 every 8 connection(s)
I20260929 19:05:24.080475 29699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:05:24 dist-test-slave-xz51 krb5kdc[29324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708724, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:24.087755 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 29699
I20260929 19:05:24.087827 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:24.088009 26939 external_mini_cluster.cc:1428] Setting key 56299ccb7ab78b623e4c470feec2ec31
I20260929 19:05:24.089661 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:24.081217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:36741 (local address 127.26.78.254:43621)
0929 19:05:24.081282 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:24.081284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:24.081482 (+ 198us) server_negotiation.cc:407] Connection header received
0929 19:05:24.082048 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:24.082049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:24.082060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:24.082102 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:24.082997 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:24.083477 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:24.084454 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:24.084631 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:24.084679 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:24.086717 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:24.086718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:24.086718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:24.086725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:24.088234 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:24.088733 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:24.088734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:24.088734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:24.088789 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:24.089059 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:24.089060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:24.089060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:24.089167 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:24.089222 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:24.089530 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 19:05:24.089599 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:05:24.089996 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:24.081198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43621 (local address 127.26.78.195:36741)
0929 19:05:24.081362 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:24.081399 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:24.081950 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:24.082157 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:24.082160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:24.082711 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:24.082916 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:24.082920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:24.083555 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:24.083556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:24.084378 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:24.084380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:24.084695 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:24.084695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:24.084813 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:24.085252 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:24.085260 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:24.086627 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:24.088330 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:24.088338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:24.088341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:24.088662 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:24.088872 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:24.088873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:24.088873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:24.089016 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:24.089269 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:24.089276 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:24.089402 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 19:05:24.089506 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 19:05:24.089699 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260929 19:05:24.090451 29828 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43621
I20260929 19:05:24.090546 29828 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:24.090734 29828 heartbeater.cc:511] Master 127.26.78.254:43621 requested a full tablet report, sending...
I20260929 19:05:24.091106 29369 ts_manager.cc:194] Registered new tserver with Master: 84e26f8275054978a548537a53aeb067 (127.26.78.195:33721)
I20260929 19:05:24.091502 29369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:36741
I20260929 19:05:24.099735 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 19:05:24.110971 29428 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:24.110044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51872 (local address 127.26.78.254:43621)
0929 19:05:24.110118 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:24.110120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:24.110192 (+ 72us) server_negotiation.cc:407] Connection header received
0929 19:05:24.110254 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:24.110255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:24.110265 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:24.110328 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:24.110870 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51872: BlockingRecv error: recv got EOF from 127.0.0.1:51872 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:24.111831 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 29431
I20260929 19:05:24.116274 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 29565
I20260929 19:05:24.120766 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 29699
I20260929 19:05:24.124859 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 29337
2026-09-29T19:05:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1845 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:24 dist-test-slave-xz51 krb5kdc[29840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:24 dist-test-slave-xz51 krb5kdc[29840](info): set up 2 sockets
Sep 29 19:05:24 dist-test-slave-xz51 krb5kdc[29840](info): commencing operation
krb5kdc: starting...
W20260929 19:05:25.217733 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:25 dist-test-slave-xz51 krb5kdc[29840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:25.338315 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:42277
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:42277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:25.418736 29853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:25.419004 29853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:25.419044 29853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:25.420221 29853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:25.420298 29853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:25.420322 29853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:25.420341 29853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:25.420360 29853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:25.422432 29853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:42277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:42277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:25.422711 29853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:25.422955 29853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:25.428035 29859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:25.428069 29861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:25.428083 29858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:25.428177 29853 server_base.cc:1034] running on GCE node
I20260929 19:05:25.428406 29853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:25.428715 29853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:25.429886 29853 hybrid_clock.cc:648] HybridClock initialized: now 1790708725429866 us; error 37 us; skew 500 ppm
Sep 29 19:05:25 dist-test-slave-xz51 krb5kdc[29840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708725, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:25.432299 29853 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:25.432737 29853 webserver.cc:460] Webserver started at http://127.26.78.254:45595/ using document root <none> and password file <none>
I20260929 19:05:25.432984 29853 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:25.433053 29853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:25.433192 29853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:25.434333 29853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "ecd2696b39fa492ebaacf9bd51fafde1"
format_stamp: "Formatted at 2026-09-29 19:05:25 on dist-test-slave-xz51"
server_key: "a850951712f629b31ee4b6581cdcb827"
server_key_iv: "b5d520dd873b81bd15693dbeeca362f7"
server_key_version: "encryptionkey@0"
I20260929 19:05:25.434681 29853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "ecd2696b39fa492ebaacf9bd51fafde1"
format_stamp: "Formatted at 2026-09-29 19:05:25 on dist-test-slave-xz51"
server_key: "a850951712f629b31ee4b6581cdcb827"
server_key_iv: "b5d520dd873b81bd15693dbeeca362f7"
server_key_version: "encryptionkey@0"
I20260929 19:05:25.435990 29853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:25.436728 29868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:25.436892 29853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:25.436990 29853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "ecd2696b39fa492ebaacf9bd51fafde1"
format_stamp: "Formatted at 2026-09-29 19:05:25 on dist-test-slave-xz51"
server_key: "a850951712f629b31ee4b6581cdcb827"
server_key_iv: "b5d520dd873b81bd15693dbeeca362f7"
server_key_version: "encryptionkey@0"
I20260929 19:05:25.437067 29853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:25.449473 29853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:25.451391 29853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:25.451550 29853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:25.455572 29853 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:42277
I20260929 19:05:25.455608 29920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:42277 every 8 connection(s)
I20260929 19:05:25.456104 29853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:25.456729 29921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:25.459060 29921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1: Bootstrap starting.
I20260929 19:05:25.459604 29921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:25.459851 29921 log.cc:824] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:25.460522 29921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1: No bootstrap required, opened a new log
I20260929 19:05:25.462260 29921 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42277 } }
I20260929 19:05:25.462414 29921 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:25.462438 29921 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd2696b39fa492ebaacf9bd51fafde1, State: Initialized, Role: FOLLOWER
I20260929 19:05:25.462586 29921 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42277 } }
I20260929 19:05:25.462663 29921 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:25.462687 29921 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:25.462719 29921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:25.462996 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 29853
I20260929 19:05:25.463088 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:25.463296 26939 external_mini_cluster.cc:1428] Setting key 827abf3d38dc039934ce9c7236f6920d
I20260929 19:05:25.464035 29921 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42277 } }
I20260929 19:05:25.464128 29921 leader_election.cc:302] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecd2696b39fa492ebaacf9bd51fafde1; no voters:
I20260929 19:05:25.464334 29921 leader_election.cc:288] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:25.464365 29926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:25.464581 29926 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [term 1 LEADER]: Becoming Leader. State: Replica: ecd2696b39fa492ebaacf9bd51fafde1, State: Running, Role: LEADER
I20260929 19:05:25.464699 29921 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:25.464708 29926 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42277 } }
I20260929 19:05:25.465251 29928 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecd2696b39fa492ebaacf9bd51fafde1. Latest consensus state: current_term: 1 leader_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42277 } } }
I20260929 19:05:25.465444 29928 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:25.465639 29927 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd2696b39fa492ebaacf9bd51fafde1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42277 } } }
I20260929 19:05:25.465746 29927 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:25.465920 29935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:25.466491 29935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:25.467971 29935 catalog_manager.cc:1347] Generated new cluster ID: eef5464aca4c41ff99e3540e71418e60
I20260929 19:05:25.468022 29935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:05:25 dist-test-slave-xz51 krb5kdc[29840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:25.475651 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:25.463834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.26.78.254:42277)
0929 19:05:25.464032 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:25.464042 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:25.464075 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:05:25.464215 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:25.464222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:25.464281 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:25.464425 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:25.465216 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:25.466699 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:25.467735 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:25.468328 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:25.468437 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:25.471223 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:25.471228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:25.471228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:25.471240 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:25.473972 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:25.474353 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:25.474356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:25.474356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:25.474479 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:25.474679 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:25.474680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:25.474680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:25.475013 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:25.475177 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:25.475258 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 19:05:25.475415 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260929 19:05:25.476598 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:05:25.477636 29935 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:25.478183 29935 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:25.481405 29935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecd2696b39fa492ebaacf9bd51fafde1: Generated new TSK 0
I20260929 19:05:25.481581 29935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:05:25.492027 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:25.488104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57982 (local address 127.26.78.254:42277)
0929 19:05:25.488164 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:25.488168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:25.488262 (+ 94us) server_negotiation.cc:407] Connection header received
0929 19:05:25.488348 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:25.488350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:25.488364 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:25.488451 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:25.488801 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:25.489603 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:25.490382 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:25.490721 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:25.490787 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:25.491504 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:25.491506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:25.491506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:25.491746 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:25.491766 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:25.491773 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:25.491836 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:25.491911 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 19:05:25.491939 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:05:25.494346 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 29853
2026-09-29T19:05:25Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1368 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:25 dist-test-slave-xz51 krb5kdc[29954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:25 dist-test-slave-xz51 krb5kdc[29954](info): set up 2 sockets
Sep 29 19:05:25 dist-test-slave-xz51 krb5kdc[29954](info): commencing operation
krb5kdc: starting...
W20260929 19:05:26.525744 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:26 dist-test-slave-xz51 krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:26.646772 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:46865
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:46865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:26.727819 29967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:26.728039 29967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:26.728067 29967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:26.729238 29967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:26.729282 29967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:26.729300 29967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:26.729338 29967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:26.729369 29967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:26.730952 29967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:46865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:46865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:26.731210 29967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:26.731456 29967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:26.736387 29973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:26.736438 29972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:26.736446 29975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:26.736515 29967 server_base.cc:1034] running on GCE node
I20260929 19:05:26.736811 29967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:26.737118 29967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:26.738278 29967 hybrid_clock.cc:648] HybridClock initialized: now 1790708726738260 us; error 50 us; skew 500 ppm
Sep 29 19:05:26 dist-test-slave-xz51 krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708726, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:26.740758 29967 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:26.741209 29967 webserver.cc:460] Webserver started at http://127.26.78.254:46601/ using document root <none> and password file <none>
I20260929 19:05:26.741410 29967 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:26.741451 29967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:26.741539 29967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:26.742623 29967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "724accd907ef4e67a79b3badc649c719"
format_stamp: "Formatted at 2026-09-29 19:05:26 on dist-test-slave-xz51"
server_key: "46008a23bf127b59d164c029dfbd2c44"
server_key_iv: "189ac522c2bbfd03610e4cbfc00bd62e"
server_key_version: "encryptionkey@0"
I20260929 19:05:26.742931 29967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "724accd907ef4e67a79b3badc649c719"
format_stamp: "Formatted at 2026-09-29 19:05:26 on dist-test-slave-xz51"
server_key: "46008a23bf127b59d164c029dfbd2c44"
server_key_iv: "189ac522c2bbfd03610e4cbfc00bd62e"
server_key_version: "encryptionkey@0"
I20260929 19:05:26.744159 29967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:05:26.744845 29982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:26.745002 29967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:26.745079 29967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "724accd907ef4e67a79b3badc649c719"
format_stamp: "Formatted at 2026-09-29 19:05:26 on dist-test-slave-xz51"
server_key: "46008a23bf127b59d164c029dfbd2c44"
server_key_iv: "189ac522c2bbfd03610e4cbfc00bd62e"
server_key_version: "encryptionkey@0"
I20260929 19:05:26.745132 29967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:26.758339 29967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:26.760645 29967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:26.760764 29967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:26.764550 29967 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:46865
I20260929 19:05:26.764619 30034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:46865 every 8 connection(s)
I20260929 19:05:26.764958 29967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:26.766008 30035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:26.768376 30035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719: Bootstrap starting.
I20260929 19:05:26.768980 30035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:26.769265 30035 log.cc:824] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:26.770026 30035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719: No bootstrap required, opened a new log
I20260929 19:05:26.771477 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 29967
I20260929 19:05:26.771561 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:26.771730 26939 external_mini_cluster.cc:1428] Setting key 6c2aa00995385173fb4eea03f597066e
I20260929 19:05:26.771683 30035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724accd907ef4e67a79b3badc649c719" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46865 } }
I20260929 19:05:26.771878 30035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:26.771924 30035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 724accd907ef4e67a79b3badc649c719, State: Initialized, Role: FOLLOWER
I20260929 19:05:26.772089 30035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724accd907ef4e67a79b3badc649c719" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46865 } }
I20260929 19:05:26.772181 30035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:26.772212 30035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:26.772274 30035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:26.773061 30035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724accd907ef4e67a79b3badc649c719" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46865 } }
I20260929 19:05:26.773177 30035 leader_election.cc:302] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 724accd907ef4e67a79b3badc649c719; no voters:
I20260929 19:05:26.773347 30035 leader_election.cc:288] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:26.773380 30040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:26.773489 30040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [term 1 LEADER]: Becoming Leader. State: Replica: 724accd907ef4e67a79b3badc649c719, State: Running, Role: LEADER
I20260929 19:05:26.773609 30035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:26.773628 30040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724accd907ef4e67a79b3badc649c719" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46865 } }
I20260929 19:05:26.774011 30040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 724accd907ef4e67a79b3badc649c719. Latest consensus state: current_term: 1 leader_uuid: "724accd907ef4e67a79b3badc649c719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724accd907ef4e67a79b3badc649c719" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46865 } } }
I20260929 19:05:26.774258 30040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:26.774669 30049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:26.774179 30041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "724accd907ef4e67a79b3badc649c719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724accd907ef4e67a79b3badc649c719" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46865 } } }
I20260929 19:05:26.775130 30041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:26.775254 30049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:26.776829 30049 catalog_manager.cc:1347] Generated new cluster ID: 3c1fac4fe249451f8eebbfa930f1b898
I20260929 19:05:26.776883 30049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:05:26 dist-test-slave-xz51 krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:26.783619 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:26.772193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.26.78.254:46865)
0929 19:05:26.772582 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:26.772589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:26.772613 (+ 24us) server_negotiation.cc:407] Connection header received
0929 19:05:26.772712 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:26.772717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:26.772758 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:26.772897 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:26.773638 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:26.775287 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:26.776410 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:26.776919 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:26.777036 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:26.779807 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:26.779812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:26.779812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:26.779825 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:26.781897 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:26.782278 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:26.782281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:26.782282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:26.782406 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:26.782606 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:26.782607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:26.782608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:26.783012 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:26.783179 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:26.783260 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 19:05:26.783409 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":114,"threads_started":1}
I20260929 19:05:26.800304 30049 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:26.800876 30049 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:26.805876 30049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 724accd907ef4e67a79b3badc649c719: Generated new TSK 0
I20260929 19:05:26.806097 30049 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:26.836663 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46865
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:26.924952 30061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:26.925155 30061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:26.925179 30061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:26.926369 30061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:26.926411 30061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:26.926438 30061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:26.927904 30061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:26.928159 30061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:26.928471 30061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:26.933650 30069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:26.933732 30066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:26.933650 30067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:26.934077 30061 server_base.cc:1034] running on GCE node
I20260929 19:05:26.934319 30061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:26.934561 30061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:26.935711 30061 hybrid_clock.cc:648] HybridClock initialized: now 1790708726935699 us; error 44 us; skew 500 ppm
Sep 29 19:05:26 dist-test-slave-xz51 krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708726, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:26.941460 30061 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:26.942050 30061 webserver.cc:460] Webserver started at http://127.26.78.193:41459/ using document root <none> and password file <none>
I20260929 19:05:26.942281 30061 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:26.942337 30061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:26.942454 30061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:26.943503 30061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "e9496fa5243b41179e1b5e6c97d8b054"
format_stamp: "Formatted at 2026-09-29 19:05:26 on dist-test-slave-xz51"
server_key: "15c64f276033bcd97703f1497d18b137"
server_key_iv: "80f3a8c92e35c672a2809ea80510dc25"
server_key_version: "encryptionkey@0"
I20260929 19:05:26.943805 30061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "e9496fa5243b41179e1b5e6c97d8b054"
format_stamp: "Formatted at 2026-09-29 19:05:26 on dist-test-slave-xz51"
server_key: "15c64f276033bcd97703f1497d18b137"
server_key_iv: "80f3a8c92e35c672a2809ea80510dc25"
server_key_version: "encryptionkey@0"
I20260929 19:05:26.945082 30061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:26.945756 30076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:26.945937 30061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:26.946023 30061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "e9496fa5243b41179e1b5e6c97d8b054"
format_stamp: "Formatted at 2026-09-29 19:05:26 on dist-test-slave-xz51"
server_key: "15c64f276033bcd97703f1497d18b137"
server_key_iv: "80f3a8c92e35c672a2809ea80510dc25"
server_key_version: "encryptionkey@0"
I20260929 19:05:26.946092 30061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:26.955113 30061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:26.956954 30061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:26.957120 30061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:26.957353 30061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:26.957664 30061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:26.957696 30061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:26.957718 30061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:26.957760 30061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:26.963496 30061 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:44579
I20260929 19:05:26.963575 30189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:44579 every 8 connection(s)
I20260929 19:05:26.963912 30061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:05:26 dist-test-slave-xz51 krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708726, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:26.972326 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30061
I20260929 19:05:26.972432 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:26.972640 26939 external_mini_cluster.cc:1428] Setting key 3fec650d4a1996f35d29db6357329b1d
I20260929 19:05:26.973450 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:26.964689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:44291 (local address 127.26.78.254:46865)
0929 19:05:26.964741 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:26.964743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:26.964965 (+ 222us) server_negotiation.cc:407] Connection header received
0929 19:05:26.965455 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:26.965456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:26.965466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:26.965515 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:26.966592 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:26.967125 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:26.968221 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:26.968426 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:26.968482 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:26.970552 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:26.970554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:26.970554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:26.970564 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:26.971934 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:26.972282 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:26.972283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:26.972283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:26.972368 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:26.972646 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:26.972647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:26.972648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:26.972788 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:26.972874 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:26.973215 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 19:05:26.973316 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:26.973573 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:26.964679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46865 (local address 127.26.78.193:44291)
0929 19:05:26.964841 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:26.964878 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:26.965372 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:26.965577 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:26.965580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:26.966251 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:26.966477 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:26.966482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:26.967202 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:26.967202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:26.968148 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:26.968151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:26.968498 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:26.968499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:26.968598 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:26.969062 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:26.969071 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:26.970470 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:26.972000 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:26.972005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:26.972007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:26.972210 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:26.972448 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:26.972449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:26.972449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:26.972579 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:26.972867 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:26.972874 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:26.973046 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 19:05:26.973157 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 19:05:26.973334 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260929 19:05:26.974080 30190 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46865
I20260929 19:05:26.974187 30190 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:26.974390 30190 heartbeater.cc:511] Master 127.26.78.254:46865 requested a full tablet report, sending...
I20260929 19:05:26.974911 29999 ts_manager.cc:194] Registered new tserver with Master: e9496fa5243b41179e1b5e6c97d8b054 (127.26.78.193:44579)
I20260929 19:05:26.975620 29999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:44291
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:26.997146 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46865
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:27.090483 30195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:27.090728 30195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:27.090767 30195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:27.092082 30195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:27.092144 30195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:27.092187 30195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:27.093716 30195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:27.094022 30195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:27.094269 30195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:27.099337 30201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:27.099345 30203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:27.099356 30200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:27.099471 30195 server_base.cc:1034] running on GCE node
I20260929 19:05:27.099808 30195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:27.100070 30195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:27.101241 30195 hybrid_clock.cc:648] HybridClock initialized: now 1790708727101225 us; error 29 us; skew 500 ppm
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708727, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:27.103926 30195 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:27.104393 30195 webserver.cc:460] Webserver started at http://127.26.78.194:39009/ using document root <none> and password file <none>
I20260929 19:05:27.104636 30195 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:27.104707 30195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:27.104862 30195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:27.106086 30195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "37bf3a57e38d48dfa7628a3360031e49"
format_stamp: "Formatted at 2026-09-29 19:05:27 on dist-test-slave-xz51"
server_key: "658fefc768646ab3256575f53a341232"
server_key_iv: "cfcdd14f538ee0c45c48cd435a3560d7"
server_key_version: "encryptionkey@0"
I20260929 19:05:27.106585 30195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "37bf3a57e38d48dfa7628a3360031e49"
format_stamp: "Formatted at 2026-09-29 19:05:27 on dist-test-slave-xz51"
server_key: "658fefc768646ab3256575f53a341232"
server_key_iv: "cfcdd14f538ee0c45c48cd435a3560d7"
server_key_version: "encryptionkey@0"
I20260929 19:05:27.107983 30195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:27.108738 30210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:27.108906 30195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:27.109004 30195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "37bf3a57e38d48dfa7628a3360031e49"
format_stamp: "Formatted at 2026-09-29 19:05:27 on dist-test-slave-xz51"
server_key: "658fefc768646ab3256575f53a341232"
server_key_iv: "cfcdd14f538ee0c45c48cd435a3560d7"
server_key_version: "encryptionkey@0"
I20260929 19:05:27.109082 30195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:27.129663 30195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:27.132158 30195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:27.132341 30195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:27.132596 30195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:27.132977 30195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:27.133037 30195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:27.133080 30195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:27.133107 30195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:27.139036 30195 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:33841
I20260929 19:05:27.139081 30323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:33841 every 8 connection(s)
I20260929 19:05:27.139412 30195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:27.142282 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30195
I20260929 19:05:27.142382 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:27.142568 26939 external_mini_cluster.cc:1428] Setting key 4fa5c5ed424e40990f4f5fdf101e3818
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708727, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:27.150403 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:27.140102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:60107 (local address 127.26.78.254:46865)
0929 19:05:27.140149 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:27.140151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:27.140384 (+ 233us) server_negotiation.cc:407] Connection header received
0929 19:05:27.140928 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:27.140929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:27.140943 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:27.140994 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:27.142042 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.142837 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.143984 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.144321 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.144394 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:27.147289 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:27.147291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:27.147291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:27.147299 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:27.148936 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:27.149358 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:27.149359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:27.149360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:27.149422 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:27.149661 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:27.149662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:27.149662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:27.149866 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:27.149933 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:27.150209 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 19:05:27.150302 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:27.150621 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:27.140091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46865 (local address 127.26.78.194:60107)
0929 19:05:27.140257 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:27.140296 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:27.140818 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:27.141068 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:27.141072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:27.141676 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:27.141945 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:27.141950 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.142943 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.142945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:27.143898 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:27.143901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.144428 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.144430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:27.144598 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:27.145176 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:27.145187 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:27.147183 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:27.149050 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:27.149056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:27.149057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:27.149271 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:27.149476 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:27.149476 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:27.149477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:27.149593 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:27.149922 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:27.149930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:27.150076 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 19:05:27.150187 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 19:05:27.150370 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
I20260929 19:05:27.151209 30324 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46865
I20260929 19:05:27.151328 30324 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:27.151541 30324 heartbeater.cc:511] Master 127.26.78.254:46865 requested a full tablet report, sending...
I20260929 19:05:27.152001 29999 ts_manager.cc:194] Registered new tserver with Master: 37bf3a57e38d48dfa7628a3360031e49 (127.26.78.194:33841)
I20260929 19:05:27.152449 29999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:60107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:27.169502 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46865
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:27.250736 30329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:27.250983 30329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:27.251024 30329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:27.252221 30329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:27.252285 30329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:27.252329 30329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:27.253790 30329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:27.254109 30329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:27.254340 30329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:27.259459 30337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:27.259460 30335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:27.259456 30334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:27.259797 30329 server_base.cc:1034] running on GCE node
I20260929 19:05:27.259984 30329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:27.260198 30329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:27.261348 30329 hybrid_clock.cc:648] HybridClock initialized: now 1790708727261337 us; error 27 us; skew 500 ppm
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708727, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:27.264098 30329 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:27.264545 30329 webserver.cc:460] Webserver started at http://127.26.78.195:36197/ using document root <none> and password file <none>
I20260929 19:05:27.264727 30329 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:27.264766 30329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:27.264863 30329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:27.265977 30329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "e9ba0984c84b4d5695f3f7e9bc85bd4e"
format_stamp: "Formatted at 2026-09-29 19:05:27 on dist-test-slave-xz51"
server_key: "023b065ba1080e6de48211cdb6a06443"
server_key_iv: "5afe8ca6b675b41d7bcf3b82e5afd906"
server_key_version: "encryptionkey@0"
I20260929 19:05:27.266278 30329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "e9ba0984c84b4d5695f3f7e9bc85bd4e"
format_stamp: "Formatted at 2026-09-29 19:05:27 on dist-test-slave-xz51"
server_key: "023b065ba1080e6de48211cdb6a06443"
server_key_iv: "5afe8ca6b675b41d7bcf3b82e5afd906"
server_key_version: "encryptionkey@0"
I20260929 19:05:27.267558 30329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:27.268319 30344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:27.268479 30329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:27.268564 30329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "e9ba0984c84b4d5695f3f7e9bc85bd4e"
format_stamp: "Formatted at 2026-09-29 19:05:27 on dist-test-slave-xz51"
server_key: "023b065ba1080e6de48211cdb6a06443"
server_key_iv: "5afe8ca6b675b41d7bcf3b82e5afd906"
server_key_version: "encryptionkey@0"
I20260929 19:05:27.268620 30329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:27.281083 30329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:27.283427 30329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:27.283583 30329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:27.283828 30329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:27.284137 30329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:27.284188 30329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:27.284221 30329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:27.284248 30329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:27.290078 30329 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:42723
I20260929 19:05:27.290132 30457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:42723 every 8 connection(s)
I20260929 19:05:27.290472 30329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:05:27.294412 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30329
I20260929 19:05:27.294510 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:27.294669 26939 external_mini_cluster.cc:1428] Setting key 28112c718b222447cea83be79c8a4e69
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708727, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:27.300040 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:27.291187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:33373 (local address 127.26.78.254:46865)
0929 19:05:27.291256 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:27.291259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:27.291494 (+ 235us) server_negotiation.cc:407] Connection header received
0929 19:05:27.291993 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:27.291994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:27.292005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:27.292074 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:27.292999 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.293485 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.294559 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.294803 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.294964 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:27.297260 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:27.297262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:27.297262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:27.297270 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:27.298673 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:27.299074 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:27.299075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:27.299075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:27.299138 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:27.299389 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:27.299390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:27.299391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:27.299514 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:27.299587 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:27.299863 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 19:05:27.299943 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:05:27.300236 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:27.291183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46865 (local address 127.26.78.195:33373)
0929 19:05:27.291358 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:27.291401 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:27.291906 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:27.292130 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:27.292133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:27.292727 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:27.292923 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:27.292927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.293561 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.293562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:27.294462 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:27.294464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.294973 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.294975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:27.295126 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:27.295684 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:27.295695 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:27.297162 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:27.298759 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:27.298766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:27.298769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:27.298996 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:27.299203 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:27.299204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:27.299204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:27.299323 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:27.299578 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:27.299584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:27.299729 (+ 145us) client_negotiation.cc:769] Sending connection context
0929 19:05:27.299840 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 19:05:27.300002 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260929 19:05:27.300639 30458 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46865
I20260929 19:05:27.300720 30458 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:27.300911 30458 heartbeater.cc:511] Master 127.26.78.254:46865 requested a full tablet report, sending...
I20260929 19:05:27.301286 29999 ts_manager.cc:194] Registered new tserver with Master: e9ba0984c84b4d5695f3f7e9bc85bd4e (127.26.78.195:42723)
I20260929 19:05:27.301707 29999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:33373
I20260929 19:05:27.306609 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708727, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708727, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:27.329869 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:27.322892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.26.78.254:46865)
0929 19:05:27.322957 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:27.322959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:27.323096 (+ 137us) server_negotiation.cc:407] Connection header received
0929 19:05:27.323189 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:27.323190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:27.323201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:27.323271 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:27.323860 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.324354 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.325097 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:27.325292 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:27.325375 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:27.327635 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:27.327636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:27.327636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:27.327643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:27.328842 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:27.329148 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:27.329149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:27.329149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:27.329198 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:27.329364 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:27.329365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:27.329365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:27.329509 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:27.329585 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:27.329682 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:05:27.329743 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260929 19:05:27.330128 29999 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34514
I20260929 19:05:27.330550 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30061
I20260929 19:05:27.335114 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30195
I20260929 19:05:27.339605 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30329
I20260929 19:05:27.343801 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 29967
2026-09-29T19:05:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1851 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[30470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[30470](info): set up 2 sockets
Sep 29 19:05:27 dist-test-slave-xz51 krb5kdc[30470](info): commencing operation
krb5kdc: starting...
W20260929 19:05:28.385915 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:28 dist-test-slave-xz51 krb5kdc[30470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:28.505220 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38111
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:38111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:28.587102 30483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:28.587356 30483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:28.587394 30483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:28.588570 30483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:28.588644 30483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:28.588670 30483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:28.588701 30483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:28.588732 30483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:28.590266 30483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:38111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:28.590544 30483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:28.590778 30483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:05:28.595765 30483 server_base.cc:1034] running on GCE node
W20260929 19:05:28.595742 30489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:28.595732 30488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:28.595718 30491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:28.596284 30483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:28.596553 30483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:28.597746 30483 hybrid_clock.cc:648] HybridClock initialized: now 1790708728597728 us; error 43 us; skew 500 ppm
Sep 29 19:05:28 dist-test-slave-xz51 krb5kdc[30470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708728, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:28.606827 30483 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:28.607417 30483 webserver.cc:460] Webserver started at http://127.26.78.254:44275/ using document root <none> and password file <none>
I20260929 19:05:28.607710 30483 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:28.607777 30483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:28.607885 30483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:28.608969 30483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "70c114edbc2741eead8c471d37673860"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "732bcc6d4f6db3bb7448b8302c4795d7"
server_key_iv: "bf314a02daa5a8ee8a2c044423f09f22"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.609277 30483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "70c114edbc2741eead8c471d37673860"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "732bcc6d4f6db3bb7448b8302c4795d7"
server_key_iv: "bf314a02daa5a8ee8a2c044423f09f22"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.610682 30483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:05:28.611430 30498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:28.611604 30483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:28.611704 30483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "70c114edbc2741eead8c471d37673860"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "732bcc6d4f6db3bb7448b8302c4795d7"
server_key_iv: "bf314a02daa5a8ee8a2c044423f09f22"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.611785 30483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:28.636090 30483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:28.638523 30483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:28.638695 30483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:28.642748 30483 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:38111
I20260929 19:05:28.642789 30550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:38111 every 8 connection(s)
I20260929 19:05:28.643338 30483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:28.644011 30551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:28.646311 30551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860: Bootstrap starting.
I20260929 19:05:28.646867 30551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:28.647107 30551 log.cc:824] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:28.647746 30551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860: No bootstrap required, opened a new log
I20260929 19:05:28.649262 30551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c114edbc2741eead8c471d37673860" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38111 } }
I20260929 19:05:28.649399 30551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:28.649422 30551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70c114edbc2741eead8c471d37673860, State: Initialized, Role: FOLLOWER
I20260929 19:05:28.649564 30551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c114edbc2741eead8c471d37673860" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38111 } }
I20260929 19:05:28.649647 30551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:28.649672 30551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:28.649706 30551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:28.650211 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30483
I20260929 19:05:28.650331 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:28.650519 26939 external_mini_cluster.cc:1428] Setting key 5901e647654799915e62921a066dbffd
I20260929 19:05:28.650746 30551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c114edbc2741eead8c471d37673860" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38111 } }
I20260929 19:05:28.650872 30551 leader_election.cc:302] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70c114edbc2741eead8c471d37673860; no voters:
I20260929 19:05:28.651041 30551 leader_election.cc:288] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:28.651271 30555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:28.651307 30551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:28.651467 30555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [term 1 LEADER]: Becoming Leader. State: Replica: 70c114edbc2741eead8c471d37673860, State: Running, Role: LEADER
I20260929 19:05:28.651567 30555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c114edbc2741eead8c471d37673860" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38111 } }
I20260929 19:05:28.651909 30555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70c114edbc2741eead8c471d37673860. Latest consensus state: current_term: 1 leader_uuid: "70c114edbc2741eead8c471d37673860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c114edbc2741eead8c471d37673860" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38111 } } }
I20260929 19:05:28.652001 30555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:28.651917 30558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70c114edbc2741eead8c471d37673860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c114edbc2741eead8c471d37673860" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38111 } } }
I20260929 19:05:28.652138 30558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860 [sys.catalog]: This master's current role is: LEADER
W20260929 19:05:28.653317 30572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:05:28.653375 30572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:05:28.653420 30565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:28.653558 30565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:28.655164 30565 catalog_manager.cc:1347] Generated new cluster ID: 11d107d5fd894ee79cef97c7310821e7
I20260929 19:05:28.655220 30565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:05:28 dist-test-slave-xz51 krb5kdc[30470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:28.662922 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:28.651028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.26.78.254:38111)
0929 19:05:28.651223 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:28.651233 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:28.651267 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:05:28.651408 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:28.651415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:28.651471 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:28.652523 (+ 1052us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:28.652647 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:28.653919 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:28.655029 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:28.655678 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:28.655798 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:28.658615 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:28.658619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:28.658620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:28.658632 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:28.661222 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:28.661658 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:28.661661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:28.661662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:28.661772 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:28.662003 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:28.662003 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:28.662004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:28.662310 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:28.662460 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:28.662582 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 19:05:28.662711 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260929 19:05:28.669045 30565 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:28.669756 30565 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:28.679060 30565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70c114edbc2741eead8c471d37673860: Generated new TSK 0
I20260929 19:05:28.679268 30565 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:28.712620 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38111
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:28.814059 30577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:28.814309 30577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:28.814368 30577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:28.815637 30577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:28.815696 30577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:28.815742 30577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:28.817265 30577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:28.817540 30577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:28.817773 30577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:28.822641 30585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:28.822641 30583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:28.822672 30582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:28.823019 30577 server_base.cc:1034] running on GCE node
I20260929 19:05:28.823195 30577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:28.823439 30577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:28.824582 30577 hybrid_clock.cc:648] HybridClock initialized: now 1790708728824564 us; error 32 us; skew 500 ppm
Sep 29 19:05:28 dist-test-slave-xz51 krb5kdc[30470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708728, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:28.827193 30577 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:28.827600 30577 webserver.cc:460] Webserver started at http://127.26.78.193:40715/ using document root <none> and password file <none>
I20260929 19:05:28.827781 30577 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:28.827857 30577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:28.827981 30577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:28.828986 30577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "e039f9a9bf134999b341b62bbe27acf7"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "312ca38406c9fe2537a7f254f3173b5d"
server_key_iv: "2aabdb805e1ceffe15baa019f60d4d26"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.829290 30577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "e039f9a9bf134999b341b62bbe27acf7"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "312ca38406c9fe2537a7f254f3173b5d"
server_key_iv: "2aabdb805e1ceffe15baa019f60d4d26"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.830651 30577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:05:28.831329 30592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:28.831503 30577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:28.831609 30577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "e039f9a9bf134999b341b62bbe27acf7"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "312ca38406c9fe2537a7f254f3173b5d"
server_key_iv: "2aabdb805e1ceffe15baa019f60d4d26"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.831684 30577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:28.844463 30577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:28.846895 30577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:28.847059 30577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:28.847290 30577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:28.847602 30577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:28.847656 30577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:28.847697 30577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:28.847723 30577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:28.853725 30577 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:32861
I20260929 19:05:28.853854 30705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:32861 every 8 connection(s)
I20260929 19:05:28.854156 30577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:05:28.858392 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30577
I20260929 19:05:28.858496 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:28.858686 26939 external_mini_cluster.cc:1428] Setting key 1b0689ae2ce3d40f1d8dd87ed93d1177
Sep 29 19:05:28 dist-test-slave-xz51 krb5kdc[30470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708728, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:28.864898 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:28.854898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:46701 (local address 127.26.78.254:38111)
0929 19:05:28.854976 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:28.854978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:28.855184 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:05:28.856132 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:28.856134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:28.856149 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:28.856204 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:28.856860 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:28.857386 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:28.858483 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:28.858810 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:28.858898 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:28.861868 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:28.861870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:28.861870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:28.861878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:28.863338 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:28.863879 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:28.863880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:28.863881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:28.863933 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:28.864215 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:28.864216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:28.864216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:28.864334 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:28.864383 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:28.864760 (+ 377us) server_negotiation.cc:299] Negotiation successful
0929 19:05:28.864831 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 19:05:28.865226 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:28.854895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38111 (local address 127.26.78.193:46701)
0929 19:05:28.855041 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:28.855083 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:28.856017 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:28.856282 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:28.856285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:28.856550 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:28.856750 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:28.856754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:28.857488 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:28.857489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:28.858388 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:28.858390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:28.858913 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:28.858914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:28.859080 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:28.859641 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:28.859652 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:28.861728 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:28.863425 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:28.863433 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:28.863435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:28.863794 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:28.863994 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:28.863995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:28.863996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:28.864149 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:28.864399 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:28.864406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:28.864598 (+ 192us) client_negotiation.cc:769] Sending connection context
0929 19:05:28.864746 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 19:05:28.864946 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260929 19:05:28.865840 30706 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38111
I20260929 19:05:28.865962 30706 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:28.866153 30706 heartbeater.cc:511] Master 127.26.78.254:38111 requested a full tablet report, sending...
I20260929 19:05:28.866683 30515 ts_manager.cc:194] Registered new tserver with Master: e039f9a9bf134999b341b62bbe27acf7 (127.26.78.193:32861)
I20260929 19:05:28.867300 30515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:46701
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:28.885442 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38111
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:28.968804 30711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:28.969024 30711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:28.969055 30711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:28.970197 30711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:28.970233 30711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:28.970259 30711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:28.971829 30711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:28.972038 30711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:28.972220 30711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:28.977126 30717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:28.977138 30716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:28.977131 30719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:28.977227 30711 server_base.cc:1034] running on GCE node
I20260929 19:05:28.977605 30711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:28.977878 30711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:28.979029 30711 hybrid_clock.cc:648] HybridClock initialized: now 1790708728979009 us; error 38 us; skew 500 ppm
Sep 29 19:05:28 dist-test-slave-xz51 krb5kdc[30470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708728, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:28.981609 30711 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:28.982151 30711 webserver.cc:460] Webserver started at http://127.26.78.194:38101/ using document root <none> and password file <none>
I20260929 19:05:28.982395 30711 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:28.982442 30711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:28.982589 30711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:28.983670 30711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "bc5cade2e3c74385a49d1f2baccd97a1"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "ab87f1aaf40b40422c585902583b3e6a"
server_key_iv: "4949e0b2ba8c6fa453c9be761e6d38b6"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.984040 30711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "bc5cade2e3c74385a49d1f2baccd97a1"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "ab87f1aaf40b40422c585902583b3e6a"
server_key_iv: "4949e0b2ba8c6fa453c9be761e6d38b6"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.985385 30711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:05:28.986248 30726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:28.986459 30711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:05:28.986549 30711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "bc5cade2e3c74385a49d1f2baccd97a1"
format_stamp: "Formatted at 2026-09-29 19:05:28 on dist-test-slave-xz51"
server_key: "ab87f1aaf40b40422c585902583b3e6a"
server_key_iv: "4949e0b2ba8c6fa453c9be761e6d38b6"
server_key_version: "encryptionkey@0"
I20260929 19:05:28.986624 30711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:29.020694 30711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:29.022725 30711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:29.022928 30711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:29.023186 30711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:29.023515 30711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:29.023568 30711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:29.023603 30711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:29.023630 30711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:29.029613 30711 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:42987
I20260929 19:05:29.029664 30839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:42987 every 8 connection(s)
I20260929 19:05:29.030054 30711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:29.030606 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30711
I20260929 19:05:29.030723 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:29.030938 26939 external_mini_cluster.cc:1428] Setting key 81addb80de216a680672732872111440
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708728, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
I20260929 19:05:29.040412 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:29.030846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:43079 (local address 127.26.78.254:38111)
0929 19:05:29.030894 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:29.030896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:29.031225 (+ 329us) server_negotiation.cc:407] Connection header received
0929 19:05:29.032299 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:29.032300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:29.032314 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:29.032396 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:29.033059 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.033575 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.034764 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.034980 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.035069 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:29.037416 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:29.037418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:29.037418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:29.037426 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:29.038890 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.039364 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.039365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.039366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.039421 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.039676 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.039677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.039678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.039789 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:29.039875 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:29.040219 (+ 344us) server_negotiation.cc:299] Negotiation successful
0929 19:05:29.040313 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:05:29.040642 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:29.030879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38111 (local address 127.26.78.194:43079)
0929 19:05:29.031044 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:29.031109 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:29.032185 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:29.032456 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:29.032460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:29.032721 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:29.032967 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:29.032972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.033686 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.033687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:29.034676 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:29.034678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.035080 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.035081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:29.035208 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:29.035834 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:29.035846 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:29.037315 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:29.038984 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:29.038991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:29.038993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:29.039273 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:29.039491 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:29.039492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:29.039492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:29.039606 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:29.039868 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:29.039875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:29.040077 (+ 202us) client_negotiation.cc:769] Sending connection context
0929 19:05:29.040196 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 19:05:29.040403 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260929 19:05:29.041136 30840 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38111
I20260929 19:05:29.041244 30840 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:29.041429 30840 heartbeater.cc:511] Master 127.26.78.254:38111 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
I20260929 19:05:29.041996 30515 ts_manager.cc:194] Registered new tserver with Master: bc5cade2e3c74385a49d1f2baccd97a1 (127.26.78.194:42987)
I20260929 19:05:29.042579 30515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:43079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:29.057087 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38111
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:29.143707 30845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:29.143932 30845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:29.143956 30845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:29.145135 30845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:29.145180 30845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:29.145232 30845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:29.146829 30845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:29.147078 30845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:29.147307 30845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:29.152494 30850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:29.152617 30851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:29.152873 30853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:29.153096 30845 server_base.cc:1034] running on GCE node
I20260929 19:05:29.153349 30845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:29.153592 30845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:29.154733 30845 hybrid_clock.cc:648] HybridClock initialized: now 1790708729154680 us; error 68 us; skew 500 ppm
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708729, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:29.157281 30845 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:29.157763 30845 webserver.cc:460] Webserver started at http://127.26.78.195:34331/ using document root <none> and password file <none>
I20260929 19:05:29.158023 30845 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:29.158092 30845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:29.158231 30845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:29.159286 30845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "a733d867f4844187846cd05897ea9c4c"
format_stamp: "Formatted at 2026-09-29 19:05:29 on dist-test-slave-xz51"
server_key: "ab88875d54bf6c46f28d90f96bf93ab2"
server_key_iv: "7f6865217cacc76814d239d6312ac3d6"
server_key_version: "encryptionkey@0"
I20260929 19:05:29.159648 30845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "a733d867f4844187846cd05897ea9c4c"
format_stamp: "Formatted at 2026-09-29 19:05:29 on dist-test-slave-xz51"
server_key: "ab88875d54bf6c46f28d90f96bf93ab2"
server_key_iv: "7f6865217cacc76814d239d6312ac3d6"
server_key_version: "encryptionkey@0"
I20260929 19:05:29.160979 30845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:29.161703 30860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:29.161908 30845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:29.162006 30845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "a733d867f4844187846cd05897ea9c4c"
format_stamp: "Formatted at 2026-09-29 19:05:29 on dist-test-slave-xz51"
server_key: "ab88875d54bf6c46f28d90f96bf93ab2"
server_key_iv: "7f6865217cacc76814d239d6312ac3d6"
server_key_version: "encryptionkey@0"
I20260929 19:05:29.162091 30845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:29.176391 30845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:29.178818 30845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:29.178980 30845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:29.179214 30845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:29.179530 30845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:29.179584 30845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:29.179626 30845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:29.179651 30845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:29.185568 30845 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:37037
I20260929 19:05:29.185608 30973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:37037 every 8 connection(s)
I20260929 19:05:29.186018 30845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708729, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:29.192188 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30845
I20260929 19:05:29.192278 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:29.192474 26939 external_mini_cluster.cc:1428] Setting key 81a2ad777e95466cd8a7bad341d31098
I20260929 19:05:29.195413 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:29.186754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:55571 (local address 127.26.78.254:38111)
0929 19:05:29.186800 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:29.186803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:29.186989 (+ 186us) server_negotiation.cc:407] Connection header received
0929 19:05:29.187905 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:29.187906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:29.187917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:29.187975 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:29.188547 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.189031 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.190124 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.190318 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.190375 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:29.192628 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:29.192630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:29.192630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:29.192638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:29.194230 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.194628 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.194629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.194630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.194689 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.194906 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.194907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.194907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.195012 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:29.195063 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:29.195282 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 19:05:29.195349 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:05:29.195647 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:29.186743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38111 (local address 127.26.78.195:55571)
0929 19:05:29.186884 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:29.186919 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:29.187817 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:29.188032 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:29.188035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:29.188282 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:29.188482 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:29.188486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.189105 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.189106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:29.190050 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:29.190053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.190390 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.190391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:29.190490 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:29.190971 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:29.190979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:29.192504 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:29.194316 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:29.194321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:29.194323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:29.194543 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:29.194750 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:29.194751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:29.194751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:29.194852 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:29.195063 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:29.195067 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:29.195176 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 19:05:29.195258 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:05:29.195394 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260929 19:05:29.196089 30974 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38111
I20260929 19:05:29.196184 30974 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:29.196362 30974 heartbeater.cc:511] Master 127.26.78.254:38111 requested a full tablet report, sending...
I20260929 19:05:29.196691 30515 ts_manager.cc:194] Registered new tserver with Master: a733d867f4844187846cd05897ea9c4c (127.26.78.195:37037)
I20260929 19:05:29.197060 30515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:55571
I20260929 19:05:29.204391 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:29.228361 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:29.220159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.26.78.193:32861)
0929 19:05:29.220322 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:29.220328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:29.220341 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:05:29.220373 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:29.220375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:29.220388 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:29.220465 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:29.220990 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.221700 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.222452 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.222772 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.222836 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:29.225108 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:29.225120 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:29.225120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:29.225130 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:29.226877 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.227253 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.227257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.227258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.227331 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.227527 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.227528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.227528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.227741 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:29.228047 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:29.228124 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 19:05:29.228209 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:29.236768 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:29.229775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.26.78.254:38111)
0929 19:05:29.229873 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:29.229875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:29.229884 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:29.229931 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:29.229932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:29.229947 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:29.230007 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:29.230461 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.230940 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.231620 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:29.231817 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:29.231912 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:29.234101 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:29.234102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:29.234103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:29.234111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:29.235538 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.235925 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.235926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.235926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.235986 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:29.236173 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:29.236174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:29.236174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:29.236339 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:29.236439 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:29.236582 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 19:05:29.236662 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 19:05:29.237008 30515 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54376
I20260929 19:05:29.237334 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30577
I20260929 19:05:29.242376 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30711
I20260929 19:05:29.246335 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30845
I20260929 19:05:29.250490 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30483
2026-09-29T19:05:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1906 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30986](info): set up 2 sockets
Sep 29 19:05:29 dist-test-slave-xz51 krb5kdc[30986](info): commencing operation
krb5kdc: starting...
W20260929 19:05:30.290000 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:30 dist-test-slave-xz51 krb5kdc[30986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:30.413002 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38289
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:38289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:30.495708 30999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:30.495940 30999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:30.495988 30999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:30.497218 30999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:30.497267 30999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:30.497282 30999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:30.497293 30999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:30.497306 30999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:30.498872 30999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:38289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:30.499089 30999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:30.499264 30999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:05:30.504418 30999 server_base.cc:1034] running on GCE node
W20260929 19:05:30.504382 31005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:30.504384 31004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:30.504372 31007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:30.504778 30999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:30.505136 30999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:30.506283 30999 hybrid_clock.cc:648] HybridClock initialized: now 1790708730506262 us; error 32 us; skew 500 ppm
Sep 29 19:05:30 dist-test-slave-xz51 krb5kdc[30986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708730, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:30.508772 30999 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:30.509203 30999 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:30.509279 30999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:30.509438 30999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:30.510592 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "3d6dd3f0565541988d7d66ecc0f13824"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "76658f0efa8d6b693f8c1f49680aa815"
server_key_iv: "510a6181e5e5f0d43de3d7827a224fde"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.510973 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "3d6dd3f0565541988d7d66ecc0f13824"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "76658f0efa8d6b693f8c1f49680aa815"
server_key_iv: "510a6181e5e5f0d43de3d7827a224fde"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.512306 30999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:30.513080 31013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:30.513249 30999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:30.513355 30999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "3d6dd3f0565541988d7d66ecc0f13824"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "76658f0efa8d6b693f8c1f49680aa815"
server_key_iv: "510a6181e5e5f0d43de3d7827a224fde"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.513442 30999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:30.527501 30999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:30.529480 30999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:30.529663 30999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:30.533808 30999 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:38289
I20260929 19:05:30.533885 31065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:38289 every 8 connection(s)
I20260929 19:05:30.534190 30999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:30.535143 31066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:30.537720 31066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824: Bootstrap starting.
I20260929 19:05:30.537994 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 30999
I20260929 19:05:30.538105 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:30.538308 26939 external_mini_cluster.cc:1428] Setting key 5c4fa524d0a7414315a635634220823f
I20260929 19:05:30.538568 31066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:30.538944 31066 log.cc:824] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:30.539690 31066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824: No bootstrap required, opened a new log
I20260929 19:05:30.542097 31066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6dd3f0565541988d7d66ecc0f13824" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38289 } }
I20260929 19:05:30.542276 31066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:30.542309 31066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6dd3f0565541988d7d66ecc0f13824, State: Initialized, Role: FOLLOWER
I20260929 19:05:30.542438 31066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6dd3f0565541988d7d66ecc0f13824" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38289 } }
I20260929 19:05:30.542503 31066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:30.542539 31066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:30.542585 31066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:30.543205 31066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6dd3f0565541988d7d66ecc0f13824" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38289 } }
I20260929 19:05:30.543322 31066 leader_election.cc:302] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d6dd3f0565541988d7d66ecc0f13824; no voters:
I20260929 19:05:30.543494 31066 leader_election.cc:288] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:30.543552 31071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:30.543720 31071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [term 1 LEADER]: Becoming Leader. State: Replica: 3d6dd3f0565541988d7d66ecc0f13824, State: Running, Role: LEADER
I20260929 19:05:30.543764 31066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:30.543825 31071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6dd3f0565541988d7d66ecc0f13824" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38289 } }
I20260929 19:05:30.545181 31072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d6dd3f0565541988d7d66ecc0f13824" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6dd3f0565541988d7d66ecc0f13824" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38289 } } }
I20260929 19:05:30.545274 31072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:30.545378 31073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d6dd3f0565541988d7d66ecc0f13824. Latest consensus state: current_term: 1 leader_uuid: "3d6dd3f0565541988d7d66ecc0f13824" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6dd3f0565541988d7d66ecc0f13824" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38289 } } }
I20260929 19:05:30.545449 31073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824 [sys.catalog]: This master's current role is: LEADER
Sep 29 19:05:30 dist-test-slave-xz51 krb5kdc[30986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:30.545728 31080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:30.546296 31080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:30.548128 31080 catalog_manager.cc:1347] Generated new cluster ID: e967d26703d0491bae5a0fc3fc371a09
I20260929 19:05:30.548179 31080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:30.550052 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:30.538849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37412 (local address 127.26.78.254:38289)
0929 19:05:30.539056 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:30.539066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:30.539102 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:05:30.539675 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:30.539683 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:30.539737 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:30.539935 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:30.540717 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.541571 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.542714 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.543127 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.543210 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:30.546165 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:30.546169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:30.546169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:30.546181 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:30.548414 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:30.548822 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:30.548825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:30.548825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:30.548923 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:30.549149 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:30.549150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:30.549150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:30.549427 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:30.549557 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:30.549705 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 19:05:30.549851 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260929 19:05:30.558027 31080 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:30.558758 31080 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:30.563809 31080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824: Generated new TSK 0
I20260929 19:05:30.564008 31080 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:30.593119 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38289
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:30.677984 31092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:30.678226 31092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:30.678273 31092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:30.679490 31092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:30.679571 31092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:30.679617 31092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:30.681180 31092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:30.681473 31092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:30.681728 31092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:30.686833 31097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:30.686952 31098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:30.687021 31092 server_base.cc:1034] running on GCE node
W20260929 19:05:30.686851 31100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:30.687399 31092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:30.687615 31092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:30.688769 31092 hybrid_clock.cc:648] HybridClock initialized: now 1790708730688743 us; error 42 us; skew 500 ppm
Sep 29 19:05:30 dist-test-slave-xz51 krb5kdc[30986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708730, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:30.691398 31092 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:30.691774 31092 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:30.691855 31092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:30.692011 31092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:30.693089 31092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "2d898daf732240138eaba0a1d29c14fa"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "19a770682c0f25659dc613aad6c5009e"
server_key_iv: "b891b66f3e71b4117a23e7899f4ba735"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.693472 31092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "2d898daf732240138eaba0a1d29c14fa"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "19a770682c0f25659dc613aad6c5009e"
server_key_iv: "b891b66f3e71b4117a23e7899f4ba735"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.694880 31092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:30.695612 31106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:30.695802 31092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:30.695905 31092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "2d898daf732240138eaba0a1d29c14fa"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "19a770682c0f25659dc613aad6c5009e"
server_key_iv: "b891b66f3e71b4117a23e7899f4ba735"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.695987 31092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:30.702878 31092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:30.704808 31092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:30.704974 31092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:30.705204 31092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:30.705518 31092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:30.705575 31092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:30.705617 31092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:30.705643 31092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:30.711946 31092 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:43853
I20260929 19:05:30.711990 31219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:43853 every 8 connection(s)
I20260929 19:05:30.712273 31092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:05:30 dist-test-slave-xz51 krb5kdc[30986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708730, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:30.718629 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 31092
I20260929 19:05:30.718710 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:30.718909 26939 external_mini_cluster.cc:1428] Setting key 338d5a4206250f4fb7ec3980fcef2ab4
I20260929 19:05:30.722365 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:30.713117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:36321 (local address 127.26.78.254:38289)
0929 19:05:30.713184 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:30.713186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:30.713286 (+ 100us) server_negotiation.cc:407] Connection header received
0929 19:05:30.713770 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:30.713771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:30.713782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:30.713929 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:30.714906 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.715421 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.716488 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.716681 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.716740 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:30.719100 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:30.719101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:30.719102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:30.719110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:30.720851 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:30.721305 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:30.721306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:30.721307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:30.721370 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:30.721627 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:30.721629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:30.721629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:30.721758 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:30.721863 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:30.722147 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 19:05:30.722240 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:30.722558 31222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:30.712988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38289 (local address 127.26.78.193:36321)
0929 19:05:30.713158 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:30.713201 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:30.713677 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:30.714010 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:30.714013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:30.714301 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:30.714804 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:30.714809 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.715511 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.715512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:30.716415 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:30.716418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.716759 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.716760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:30.716860 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:30.717348 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:30.717356 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:30.718998 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:30.720940 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:30.720946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:30.720948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:30.721216 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:30.721446 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:30.721447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:30.721447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:30.721557 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:30.721847 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:30.721855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:30.722002 (+ 147us) client_negotiation.cc:769] Sending connection context
0929 19:05:30.722112 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 19:05:30.722271 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260929 19:05:30.723090 31220 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38289
I20260929 19:05:30.723205 31220 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:30.723404 31220 heartbeater.cc:511] Master 127.26.78.254:38289 requested a full tablet report, sending...
I20260929 19:05:30.723949 31030 ts_manager.cc:194] Registered new tserver with Master: 2d898daf732240138eaba0a1d29c14fa (127.26.78.193:43853)
I20260929 19:05:30.724601 31030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:36321
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:30.747112 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38289
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:30.828758 31225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:30.828984 31225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:30.829007 31225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:30.830232 31225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:30.830277 31225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:30.830305 31225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:30.831892 31225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:30.832163 31225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:30.832410 31225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:05:30.837599 31225 server_base.cc:1034] running on GCE node
W20260929 19:05:30.837546 31230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:30.837572 31231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:30.837569 31233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:30.838057 31225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:30.838326 31225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:30.839505 31225 hybrid_clock.cc:648] HybridClock initialized: now 1790708730839483 us; error 36 us; skew 500 ppm
Sep 29 19:05:30 dist-test-slave-xz51 krb5kdc[30986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708730, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:30.842074 31225 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:30.842375 31225 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:30.842419 31225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:30.842507 31225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:30.843580 31225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "735fd42ea14a4fecbcdf27af8dd010c7"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "5b9636de094ca459b55edcbaafef7f8e"
server_key_iv: "5fd2ad0b1056b05297e559cbda261bff"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.843952 31225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "735fd42ea14a4fecbcdf27af8dd010c7"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "5b9636de094ca459b55edcbaafef7f8e"
server_key_iv: "5fd2ad0b1056b05297e559cbda261bff"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.845337 31225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:05:30.846230 31239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:30.846431 31225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:05:30.846516 31225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "735fd42ea14a4fecbcdf27af8dd010c7"
format_stamp: "Formatted at 2026-09-29 19:05:30 on dist-test-slave-xz51"
server_key: "5b9636de094ca459b55edcbaafef7f8e"
server_key_iv: "5fd2ad0b1056b05297e559cbda261bff"
server_key_version: "encryptionkey@0"
I20260929 19:05:30.846616 31225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:30.863907 31225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:30.866348 31225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:30.866468 31225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:30.866668 31225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:30.866952 31225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:30.866983 31225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:30.867005 31225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:30.867018 31225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:30.872848 31225 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:35305
I20260929 19:05:30.873016 31352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:35305 every 8 connection(s)
I20260929 19:05:30.873219 31225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:05:30 dist-test-slave-xz51 krb5kdc[30986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708730, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:30.882190 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 31225
I20260929 19:05:30.882293 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:30.882499 26939 external_mini_cluster.cc:1428] Setting key 71bc1cf423668e739f74f69085c555a4
I20260929 19:05:30.882766 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:30.874045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:53823 (local address 127.26.78.254:38289)
0929 19:05:30.874112 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:30.874114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:30.874347 (+ 233us) server_negotiation.cc:407] Connection header received
0929 19:05:30.874805 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:30.874806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:30.874817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:30.874885 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:30.875954 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.876446 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.877407 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.877591 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.877649 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:30.879897 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:30.879899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:30.879899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:30.879905 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:30.881334 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:30.881684 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:30.881685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:30.881685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:30.881735 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:30.882038 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:30.882039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:30.882039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:30.882145 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:30.882198 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:30.882566 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 19:05:30.882647 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:05:30.882972 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:30.874044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38289 (local address 127.26.78.194:53823)
0929 19:05:30.874206 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:30.874244 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:30.874717 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:30.874975 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:30.874978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:30.875242 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:30.875866 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:30.875871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.876526 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.876527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:30.877322 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:30.877324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:30.877668 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:30.877669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:30.877770 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:30.878396 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:30.878412 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:30.879808 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:30.881407 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:30.881412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:30.881413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:30.881619 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:30.881792 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:30.881792 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:30.881793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:30.881960 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:30.882213 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:30.882219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:30.882397 (+ 178us) client_negotiation.cc:769] Sending connection context
0929 19:05:30.882529 (+ 132us) client_negotiation.cc:240] Negotiation successful
0929 19:05:30.882699 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260929 19:05:30.883608 31353 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38289
I20260929 19:05:30.883699 31353 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:30.883864 31353 heartbeater.cc:511] Master 127.26.78.254:38289 requested a full tablet report, sending...
I20260929 19:05:30.884253 31030 ts_manager.cc:194] Registered new tserver with Master: 735fd42ea14a4fecbcdf27af8dd010c7 (127.26.78.194:35305)
I20260929 19:05:30.884682 31030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:53823
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:30.909158 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38289
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:30.993443 31358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:30.993645 31358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:30.993677 31358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:30.994947 31358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:30.994992 31358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:30.995054 31358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:30.996584 31358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:30.996858 31358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:30.997118 31358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:31.002456 31363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.002449 31366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.002453 31364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.002808 31358 server_base.cc:1034] running on GCE node
I20260929 19:05:31.003049 31358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:31.003295 31358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:31.004457 31358 hybrid_clock.cc:648] HybridClock initialized: now 1790708731004451 us; error 37 us; skew 500 ppm
Sep 29 19:05:31 dist-test-slave-xz51 krb5kdc[30986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708731, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:31.007148 31358 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:31.007509 31358 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:31.007586 31358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:31.007737 31358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:31.008848 31358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "ba44af6b0bb045cda2d71f39234503be"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "7522e180a0f5d4ed74195dacddc61266"
server_key_iv: "0a0d932b26baa5befbd04cd810c6991d"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.009222 31358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "ba44af6b0bb045cda2d71f39234503be"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "7522e180a0f5d4ed74195dacddc61266"
server_key_iv: "0a0d932b26baa5befbd04cd810c6991d"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.010607 31358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:31.011390 31372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.011560 31358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:31.011660 31358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "ba44af6b0bb045cda2d71f39234503be"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "7522e180a0f5d4ed74195dacddc61266"
server_key_iv: "0a0d932b26baa5befbd04cd810c6991d"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.011792 31358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:31.025575 31358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:31.028127 31358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:31.028312 31358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:31.028609 31358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:31.029022 31358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:31.029071 31358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.029112 31358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:31.029141 31358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.035111 31358 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:43657
I20260929 19:05:31.035181 31485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:43657 every 8 connection(s)
I20260929 19:05:31.035449 31358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:05:31 dist-test-slave-xz51 krb5kdc[30986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708731, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:31.044260 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 31358
I20260929 19:05:31.044351 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:31.044569 26939 external_mini_cluster.cc:1428] Setting key 5f08cbaa8adffec75e337786f7ec384c
I20260929 19:05:31.045111 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.036176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:50931 (local address 127.26.78.254:38289)
0929 19:05:31.036266 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.036269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.036463 (+ 194us) server_negotiation.cc:407] Connection header received
0929 19:05:31.037013 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.037015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.037027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.037066 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.038059 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.038572 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.039604 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.039812 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.039905 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.042196 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.042198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.042198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:31.042205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.043463 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:31.043932 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:31.043933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:31.043934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:31.043995 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:31.044269 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:31.044270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:31.044271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:31.044442 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.044504 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.044874 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.044990 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:05:31.045281 31488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.036146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38289 (local address 127.26.78.195:50931)
0929 19:05:31.036325 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.036370 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.036897 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.037143 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.037147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.037418 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.037953 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.037958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.038686 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.038687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.039506 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.039508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.039913 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.039914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.040045 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.040624 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:31.040633 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.042090 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.043552 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:31.043559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:31.043562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:31.043855 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:31.044074 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:31.044075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:31.044075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:31.044186 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:31.044505 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.044512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.044683 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.044815 (+ 132us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.045026 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260929 19:05:31.045902 31486 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38289
I20260929 19:05:31.045996 31486 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:31.046173 31486 heartbeater.cc:511] Master 127.26.78.254:38289 requested a full tablet report, sending...
I20260929 19:05:31.046530 31030 ts_manager.cc:194] Registered new tserver with Master: ba44af6b0bb045cda2d71f39234503be (127.26.78.195:43657)
I20260929 19:05:31.046911 31030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:50931
I20260929 19:05:31.056751 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:05:31.064249 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.058155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.26.78.254:38289)
0929 19:05:31.058226 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.058228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.058266 (+ 38us) server_negotiation.cc:407] Connection header received
0929 19:05:31.058335 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.058335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.058344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.058396 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.058956 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.059445 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.060143 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.060324 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.060375 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.061548 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.061549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.061549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:31.061555 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.063026 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:31.063403 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:31.063404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:31.063404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:31.063457 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:31.063654 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:31.063655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:31.063655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:31.063870 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.063983 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.064091 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.064161 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:31.065879 31030 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:31.067492 31030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:31.073948 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.069808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:43657 (local address 127.0.0.1:58506)
0929 19:05:31.070197 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.070209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.070267 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.070646 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.070647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.070648 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:31.070683 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.070685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.071667 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.071668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.072687 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.072690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.073615 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.073616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.073696 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.073712 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.073786 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.073865 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":5,"thread_start_us":48,"threads_started":1}
I20260929 19:05:31.074127 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.069808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43853 (local address 127.0.0.1:59004)
0929 19:05:31.069998 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.070041 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.070111 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.070386 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.070389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.070392 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:31.070472 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.070475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.071489 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.071490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.072826 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.072829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.073751 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.073752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.073870 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.073875 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.073927 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.073968 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260929 19:05:31.074014 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.069984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58506 (local address 127.26.78.195:43657)
0929 19:05:31.070504 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.070509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.070518 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:31.070555 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.070557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.070573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.070650 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:31.070727 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.071536 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.072774 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.073519 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.073601 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.073667 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.073869 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.073925 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":86,"threads_started":1}
I20260929 19:05:31.074162 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.069966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59004 (local address 127.26.78.193:43853)
0929 19:05:31.070191 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.070197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.070210 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:05:31.070253 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.070255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.070273 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.070355 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:31.070568 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.071394 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.072915 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.073606 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.073665 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.073712 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.074014 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.074068 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":105,"threads_started":1}
I20260929 19:05:31.074851 31420 tablet_service.cc:1467] Processing CreateTablet for tablet 1eef35a387114e5c978b51253d647673 (DEFAULT_TABLE table=test-table [id=cc1588459bde45a7aea89356819aa6a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:31.074879 31154 tablet_service.cc:1467] Processing CreateTablet for tablet 1eef35a387114e5c978b51253d647673 (DEFAULT_TABLE table=test-table [id=cc1588459bde45a7aea89356819aa6a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:31.075229 31420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eef35a387114e5c978b51253d647673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.075229 31154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eef35a387114e5c978b51253d647673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.075897 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.070025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:35305 (local address 127.0.0.1:55562)
0929 19:05:31.070434 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.070466 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.070515 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.071895 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.071896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.071898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:31.071953 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.071954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.073107 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.073108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.074000 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.074003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.075623 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.075625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.075703 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.075707 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.075770 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.075817 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":70,"threads_started":1}
I20260929 19:05:31.075919 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.070248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.26.78.194:35305)
0929 19:05:31.071720 (+ 1472us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.071725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.071734 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:31.071770 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.071772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.071786 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.071873 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:31.072021 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.073013 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.074722 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.075529 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.075608 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.075664 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.075798 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.075840 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":102,"threads_started":1}
I20260929 19:05:31.076685 31287 tablet_service.cc:1467] Processing CreateTablet for tablet 1eef35a387114e5c978b51253d647673 (DEFAULT_TABLE table=test-table [id=cc1588459bde45a7aea89356819aa6a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:31.077011 31287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eef35a387114e5c978b51253d647673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.078442 31503 tablet_bootstrap.cc:492] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Bootstrap starting.
I20260929 19:05:31.079111 31503 tablet_bootstrap.cc:654] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.079629 31503 log.cc:824] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.080144 31504 tablet_bootstrap.cc:492] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa: Bootstrap starting.
I20260929 19:05:31.080437 31503 tablet_bootstrap.cc:492] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: No bootstrap required, opened a new log
I20260929 19:05:31.080520 31503 ts_tablet_manager.cc:1397] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:31.080744 31504 tablet_bootstrap.cc:654] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.081274 31504 log.cc:824] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.082047 31504 tablet_bootstrap.cc:492] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa: No bootstrap required, opened a new log
I20260929 19:05:31.082098 31504 ts_tablet_manager.cc:1397] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:05:31.082392 31503 raft_consensus.cc:348] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.082547 31503 raft_consensus.cc:374] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.082590 31503 raft_consensus.cc:729] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba44af6b0bb045cda2d71f39234503be, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.082706 31503 consensus_queue.cc:260] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.083029 31503 ts_tablet_manager.cc:1428] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:05:31.083034 31486 heartbeater.cc:503] Master 127.26.78.254:38289 was elected leader, sending a full tablet report...
I20260929 19:05:31.080050 31506 tablet_bootstrap.cc:492] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Bootstrap starting.
I20260929 19:05:31.083719 31506 tablet_bootstrap.cc:654] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.084014 31504 raft_consensus.cc:348] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.084177 31504 raft_consensus.cc:374] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.084221 31504 raft_consensus.cc:729] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d898daf732240138eaba0a1d29c14fa, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.084367 31506 log.cc:824] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.084345 31504 consensus_queue.cc:260] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.084636 31504 ts_tablet_manager.cc:1428] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:05:31.084652 31220 heartbeater.cc:503] Master 127.26.78.254:38289 was elected leader, sending a full tablet report...
I20260929 19:05:31.085229 31506 tablet_bootstrap.cc:492] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: No bootstrap required, opened a new log
I20260929 19:05:31.085307 31506 ts_tablet_manager.cc:1397] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 19:05:31.086830 31506 raft_consensus.cc:348] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.086964 31506 raft_consensus.cc:374] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.086985 31506 raft_consensus.cc:729] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735fd42ea14a4fecbcdf27af8dd010c7, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.087122 31506 consensus_queue.cc:260] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.087388 31506 ts_tablet_manager.cc:1428] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:05:31.087436 31353 heartbeater.cc:503] Master 127.26.78.254:38289 was elected leader, sending a full tablet report...
W20260929 19:05:31.123797 31354 tablet.cc:2369] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:05:31.180138 31511 raft_consensus.cc:482] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:31.180281 31511 raft_consensus.cc:504] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.180806 31511 leader_election.cc:288] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d898daf732240138eaba0a1d29c14fa (127.26.78.193:43853), ba44af6b0bb045cda2d71f39234503be (127.26.78.195:43657)
I20260929 19:05:31.184211 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.181170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:58897 (local address 127.26.78.193:43853)
0929 19:05:31.181242 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.181244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.181254 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:31.181285 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.181286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.181296 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.181354 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:31.181643 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.182240 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.183388 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.183782 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.183872 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.183880 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.184090 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.184135 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:05:31.184238 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.181158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:54717 (local address 127.26.78.195:43657)
0929 19:05:31.181236 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.181238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.181521 (+ 283us) server_negotiation.cc:407] Connection header received
0929 19:05:31.181552 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.181553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.181565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.181677 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:31.181869 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.182453 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.183451 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.183825 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.183952 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.183960 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.184131 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.184173 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 19:05:31.184238 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.181048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43853 (local address 127.26.78.194:58897)
0929 19:05:31.181096 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.181109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.181165 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.181380 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.181382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.181384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:31.181450 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.181452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.182327 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.182329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.183274 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.183276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.183913 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.183915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.184002 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.184006 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.184058 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.184118 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 19:05:31.184242 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.181048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:43657 (local address 127.26.78.194:54717)
0929 19:05:31.181409 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.181419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.181471 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.181658 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.181659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.181662 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:31.181737 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.181738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.182558 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.182559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.183285 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:31.183287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.183962 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.183963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:31.184023 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.184026 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.184107 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.184143 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":170,"thread_start_us":113,"threads_started":1}
I20260929 19:05:31.184643 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef35a387114e5c978b51253d647673" candidate_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d898daf732240138eaba0a1d29c14fa" is_pre_election: true
I20260929 19:05:31.184859 31174 raft_consensus.cc:2454] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 735fd42ea14a4fecbcdf27af8dd010c7 in term 0.
I20260929 19:05:31.184899 31440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef35a387114e5c978b51253d647673" candidate_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba44af6b0bb045cda2d71f39234503be" is_pre_election: true
I20260929 19:05:31.185050 31440 raft_consensus.cc:2454] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 735fd42ea14a4fecbcdf27af8dd010c7 in term 0.
I20260929 19:05:31.185129 31241 leader_election.cc:302] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d898daf732240138eaba0a1d29c14fa, 735fd42ea14a4fecbcdf27af8dd010c7; no voters:
I20260929 19:05:31.185263 31511 raft_consensus.cc:2781] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:31.185309 31511 raft_consensus.cc:482] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:31.185335 31511 raft_consensus.cc:3037] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.186064 31511 raft_consensus.cc:504] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.186230 31511 leader_election.cc:288] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [CANDIDATE]: Term 1 election: Requested vote from peers 2d898daf732240138eaba0a1d29c14fa (127.26.78.193:43853), ba44af6b0bb045cda2d71f39234503be (127.26.78.195:43657)
I20260929 19:05:31.186404 31440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef35a387114e5c978b51253d647673" candidate_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba44af6b0bb045cda2d71f39234503be"
I20260929 19:05:31.186419 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef35a387114e5c978b51253d647673" candidate_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d898daf732240138eaba0a1d29c14fa"
I20260929 19:05:31.186496 31440 raft_consensus.cc:3037] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.186496 31174 raft_consensus.cc:3037] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.187399 31440 raft_consensus.cc:2454] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 735fd42ea14a4fecbcdf27af8dd010c7 in term 1.
I20260929 19:05:31.187399 31174 raft_consensus.cc:2454] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 735fd42ea14a4fecbcdf27af8dd010c7 in term 1.
I20260929 19:05:31.187595 31241 leader_election.cc:302] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d898daf732240138eaba0a1d29c14fa, 735fd42ea14a4fecbcdf27af8dd010c7; no voters:
I20260929 19:05:31.187700 31511 raft_consensus.cc:2781] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:31.187860 31511 raft_consensus.cc:686] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 1 LEADER]: Becoming Leader. State: Replica: 735fd42ea14a4fecbcdf27af8dd010c7, State: Running, Role: LEADER
I20260929 19:05:31.187940 31511 consensus_queue.cc:237] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } }
I20260929 19:05:31.188879 31029 catalog_manager.cc:5495] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 735fd42ea14a4fecbcdf27af8dd010c7 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "735fd42ea14a4fecbcdf27af8dd010c7" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 } health_report { overall_health: UNKNOWN } } }
W20260929 19:05:31.213001 31221 tablet.cc:2369] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:05:31.218747 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.216361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.26.78.194:35305)
0929 19:05:31.216404 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.216406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.216415 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:31.216456 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.216457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.216475 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.216547 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:31.216667 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.217301 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.218049 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.218267 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.218336 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.218460 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:31.218567 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:31.218633 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.218651 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.218667 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:05:31.220806 31174 raft_consensus.cc:1261] T 1eef35a387114e5c978b51253d647673 P 2d898daf732240138eaba0a1d29c14fa [term 1 FOLLOWER]: Refusing update from remote peer 735fd42ea14a4fecbcdf27af8dd010c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:31.220893 31440 raft_consensus.cc:1261] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 1 FOLLOWER]: Refusing update from remote peer 735fd42ea14a4fecbcdf27af8dd010c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:31.221191 31513 consensus_queue.cc:1035] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d898daf732240138eaba0a1d29c14fa" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:31.221323 31511 consensus_queue.cc:1035] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba44af6b0bb045cda2d71f39234503be" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:31.223781 31516 mvcc.cc:205] Tried to move back new op lower bound from 7334742963077173248 to 7334742962946428928. Current Snapshot: MvccSnapshot[applied={T|T < 7334742963077173248}]
I20260929 19:05:31.224676 31518 mvcc.cc:205] Tried to move back new op lower bound from 7334742963077173248 to 7334742962946428928. Current Snapshot: MvccSnapshot[applied={T|T < 7334742963077173248}]
I20260929 19:05:31.227097 31029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:31.227200 31029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:31.227999 31029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d6dd3f0565541988d7d66ecc0f13824: Sending DeleteTablet for 3 replicas of tablet 1eef35a387114e5c978b51253d647673
I20260929 19:05:31.228349 31154 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eef35a387114e5c978b51253d647673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:31 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:59004
I20260929 19:05:31.228363 31420 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eef35a387114e5c978b51253d647673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:31 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:58506
I20260929 19:05:31.228559 31287 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eef35a387114e5c978b51253d647673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:31 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:55562
I20260929 19:05:31.228843 31527 tablet_replica.cc:330] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: stopping tablet replica
I20260929 19:05:31.228961 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 31092
I20260929 19:05:31.228986 31527 raft_consensus.cc:2229] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:31.229097 31527 raft_consensus.cc:2258] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:31.229319 31528 tablet_replica.cc:330] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: stopping tablet replica
I20260929 19:05:31.229400 31528 raft_consensus.cc:2229] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:31.229511 31528 raft_consensus.cc:2258] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:31.229979 31528 ts_tablet_manager.cc:1905] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:31.230873 31527 ts_tablet_manager.cc:1905] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:31.231804 31528 ts_tablet_manager.cc:1918] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:31.231875 31528 log.cc:1196] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/1eef35a387114e5c978b51253d647673
I20260929 19:05:31.232168 31528 ts_tablet_manager.cc:1939] T 1eef35a387114e5c978b51253d647673 P 735fd42ea14a4fecbcdf27af8dd010c7: Deleting consensus metadata
I20260929 19:05:31.232638 31527 ts_tablet_manager.cc:1918] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:31.232688 31527 log.cc:1196] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/1eef35a387114e5c978b51253d647673
I20260929 19:05:31.232896 31527 ts_tablet_manager.cc:1939] T 1eef35a387114e5c978b51253d647673 P ba44af6b0bb045cda2d71f39234503be: Deleting consensus metadata
I20260929 19:05:31.233443 31015 catalog_manager.cc:4841] TS 735fd42ea14a4fecbcdf27af8dd010c7 (127.26.78.194:35305): tablet 1eef35a387114e5c978b51253d647673 (table test-table [id=cc1588459bde45a7aea89356819aa6a2]) successfully deleted
I20260929 19:05:31.233563 31017 catalog_manager.cc:4841] TS ba44af6b0bb045cda2d71f39234503be (127.26.78.195:43657): tablet 1eef35a387114e5c978b51253d647673 (table test-table [id=cc1588459bde45a7aea89356819aa6a2]) successfully deleted
W20260929 19:05:31.237598 31015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:43853 (error 108)
I20260929 19:05:31.237737 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 31225
I20260929 19:05:31.238704 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.238546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43853 (local address 127.0.0.1:59014)
0929 19:05:31.238618 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.238668 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 19:05:31.238828 31015 catalog_manager.cc:4584] TS 2d898daf732240138eaba0a1d29c14fa (127.26.78.193:43853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1eef35a387114e5c978b51253d647673: Network error: Client connection negotiation failed: client connection to 127.26.78.193:43853: connect: Connection refused (error 111)
I20260929 19:05:31.242873 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 31358
I20260929 19:05:31.247339 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 30999
2026-09-29T19:05:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1996 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T19:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:31Z Disabled control of system clock
I20260929 19:05:31.274359 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:46745
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:46745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:05:31.356205 31533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:31.356433 31533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:31.356457 31533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:31.357702 31533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:31.357753 31533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:31.357769 31533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:31.357815 31533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:31.357880 31533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:31.359359 31533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:46745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:46745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:31.359663 31533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:31.359990 31533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:31.365139 31541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.365157 31538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.365242 31533 server_base.cc:1034] running on GCE node
W20260929 19:05:31.365155 31539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.365665 31533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:31.366009 31533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:31.367182 31533 hybrid_clock.cc:648] HybridClock initialized: now 1790708731367162 us; error 51 us; skew 500 ppm
I20260929 19:05:31.368397 31533 webserver.cc:460] Webserver started at http://127.26.78.254:41919/ using document root <none> and password file <none>
I20260929 19:05:31.368634 31533 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:31.368705 31533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:31.368876 31533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:31.370024 31533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "941064e096ae444fa78620b0bf8054ad"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "0d1542e4b393e693d780588a622dab3d"
server_key_iv: "68991dbb377f0c5da6de431a40affa52"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.370391 31533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "941064e096ae444fa78620b0bf8054ad"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "0d1542e4b393e693d780588a622dab3d"
server_key_iv: "68991dbb377f0c5da6de431a40affa52"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.371712 31533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:31.372553 31547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.372812 31533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:31.372920 31533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "941064e096ae444fa78620b0bf8054ad"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "0d1542e4b393e693d780588a622dab3d"
server_key_iv: "68991dbb377f0c5da6de431a40affa52"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.372994 31533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:31.376364 31533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:31.376652 31533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:31.376786 31533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:31.380764 31533 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:46745
I20260929 19:05:31.380839 31599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:46745 every 8 connection(s)
I20260929 19:05:31.381160 31533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:31.381778 31600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.384548 31600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad: Bootstrap starting.
I20260929 19:05:31.385061 31600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.385309 31600 log.cc:824] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.386049 31600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad: No bootstrap required, opened a new log
I20260929 19:05:31.387606 31600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "941064e096ae444fa78620b0bf8054ad" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46745 } }
I20260929 19:05:31.387744 31600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.387799 31600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 941064e096ae444fa78620b0bf8054ad, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.387916 31600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "941064e096ae444fa78620b0bf8054ad" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46745 } }
I20260929 19:05:31.387969 31600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:31.387988 31600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:31.388015 31600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.388518 31600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "941064e096ae444fa78620b0bf8054ad" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46745 } }
I20260929 19:05:31.388608 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 31533
I20260929 19:05:31.388652 31600 leader_election.cc:302] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 941064e096ae444fa78620b0bf8054ad; no voters:
I20260929 19:05:31.388728 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:31.388818 31600 leader_election.cc:288] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:31.388849 31603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:31.388916 26939 external_mini_cluster.cc:1428] Setting key 273f68ce99b9ccb9fdaa72a048078117
I20260929 19:05:31.388969 31603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [term 1 LEADER]: Becoming Leader. State: Replica: 941064e096ae444fa78620b0bf8054ad, State: Running, Role: LEADER
I20260929 19:05:31.389050 31603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "941064e096ae444fa78620b0bf8054ad" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46745 } }
I20260929 19:05:31.389083 31600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:31.389374 31604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "941064e096ae444fa78620b0bf8054ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "941064e096ae444fa78620b0bf8054ad" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46745 } } }
I20260929 19:05:31.389482 31604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:31.389734 31605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 941064e096ae444fa78620b0bf8054ad. Latest consensus state: current_term: 1 leader_uuid: "941064e096ae444fa78620b0bf8054ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "941064e096ae444fa78620b0bf8054ad" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 46745 } } }
I20260929 19:05:31.390298 31605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:31.390489 31609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:31.391522 31609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:31.391883 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.389763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.26.78.254:46745)
0929 19:05:31.390271 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.390281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.390320 (+ 39us) server_negotiation.cc:407] Connection header received
0929 19:05:31.390449 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.390457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.390506 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.390706 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.391386 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.391391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.391391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.391395 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.391443 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.391454 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.391528 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.391654 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.391719 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":305,"threads_started":1}
I20260929 19:05:31.393040 31609 catalog_manager.cc:1347] Generated new cluster ID: d2c95577479e454d951e5b0df772852e
I20260929 19:05:31.393101 31609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:31.399243 31609 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:31.399417 31609 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:31.405465 31609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad: Generated new TSK 0
I20260929 19:05:31.405674 31609 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:05:31.410111 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46745
--builtin_ntp_servers=127.26.78.212:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:05:31.491304 31624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:31.491564 31624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:31.491603 31624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:31.492776 31624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:31.492842 31624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:31.492887 31624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:31.494318 31624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:31.494582 31624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:31.494879 31624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:31.499924 31629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.500056 31630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.499948 31632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.500280 31624 server_base.cc:1034] running on GCE node
I20260929 19:05:31.500523 31624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:31.500795 31624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:31.501957 31624 hybrid_clock.cc:648] HybridClock initialized: now 1790708731501941 us; error 27 us; skew 500 ppm
I20260929 19:05:31.503422 31624 webserver.cc:460] Webserver started at http://127.26.78.193:46041/ using document root <none> and password file <none>
I20260929 19:05:31.503706 31624 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:31.503770 31624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:31.503912 31624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:31.505314 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "56079e211ede49c3861fd3781358755f"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "fec806242666443e1cf9f9eb1d3ada6e"
server_key_iv: "44c82f5d3a971a653f99d3b60295b66b"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.505803 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "56079e211ede49c3861fd3781358755f"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "fec806242666443e1cf9f9eb1d3ada6e"
server_key_iv: "44c82f5d3a971a653f99d3b60295b66b"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.507563 31624 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260929 19:05:31.508502 31638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.508706 31624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:31.508823 31624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "56079e211ede49c3861fd3781358755f"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "fec806242666443e1cf9f9eb1d3ada6e"
server_key_iv: "44c82f5d3a971a653f99d3b60295b66b"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.508900 31624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:31.512212 31624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:31.512514 31624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:31.512653 31624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:31.512871 31624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:31.513180 31624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:31.513235 31624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.513275 31624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:31.513309 31624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.518849 31624 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:38521
I20260929 19:05:31.518927 31751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:38521 every 8 connection(s)
I20260929 19:05:31.519254 31624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:05:31.522156 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.519989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:38583 (local address 127.26.78.254:46745)
0929 19:05:31.520031 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.520033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.520319 (+ 286us) server_negotiation.cc:407] Connection header received
0929 19:05:31.520731 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.520732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.520738 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.520795 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.521937 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.521939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.521939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.521943 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.521957 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.521960 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.522010 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.522091 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.522104 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:31.522331 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.519963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46745 (local address 127.26.78.193:38583)
0929 19:05:31.520165 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.520205 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.520652 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.520861 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.520864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.520872 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.521726 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.521727 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.521755 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.521781 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.522021 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.522025 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.522027 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.522087 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.522134 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260929 19:05:31.522831 31752 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46745
I20260929 19:05:31.522933 31752 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:31.523077 31752 heartbeater.cc:511] Master 127.26.78.254:46745 requested a full tablet report, sending...
I20260929 19:05:31.523557 31564 ts_manager.cc:194] Registered new tserver with Master: 56079e211ede49c3861fd3781358755f (127.26.78.193:38521)
I20260929 19:05:31.524714 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 31624
I20260929 19:05:31.524814 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:31.524979 26939 external_mini_cluster.cc:1428] Setting key d4e22c0e0c4c6e1436d3d3c13710f044
I20260929 19:05:31.525741 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46745
--builtin_ntp_servers=127.26.78.212:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:05:31.606783 31755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:31.607070 31755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:31.607103 31755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:31.608711 31755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:31.608760 31755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:31.608805 31755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:31.610728 31755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:31.610977 31755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:31.611296 31755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:31.616608 31760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.616608 31761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.617218 31755 server_base.cc:1034] running on GCE node
W20260929 19:05:31.617425 31763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.617767 31755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:31.618104 31755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:31.619269 31755 hybrid_clock.cc:648] HybridClock initialized: now 1790708731619243 us; error 39 us; skew 500 ppm
I20260929 19:05:31.620617 31755 webserver.cc:460] Webserver started at http://127.26.78.194:40519/ using document root <none> and password file <none>
I20260929 19:05:31.620818 31755 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:31.620860 31755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:31.620950 31755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:31.622071 31755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "8a3ac664729942a78f9d8bcb48c30851"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "9aae4e11d6cc33f30832c8bbdbaf71ee"
server_key_iv: "70028033935536ceff24ea3ba2039dec"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.622385 31755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "8a3ac664729942a78f9d8bcb48c30851"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "9aae4e11d6cc33f30832c8bbdbaf71ee"
server_key_iv: "70028033935536ceff24ea3ba2039dec"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.623595 31755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:31.624248 31769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.624488 31755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:31.624563 31755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "8a3ac664729942a78f9d8bcb48c30851"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "9aae4e11d6cc33f30832c8bbdbaf71ee"
server_key_iv: "70028033935536ceff24ea3ba2039dec"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.624660 31755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:31.628178 31755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:31.628453 31755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:31.628595 31755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:31.628814 31755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:31.629137 31755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:31.629195 31755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.629236 31755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:31.629266 31755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.635129 31755 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:34543
I20260929 19:05:31.635241 31882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:34543 every 8 connection(s)
I20260929 19:05:31.635560 31755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:31.638641 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.636323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:60497 (local address 127.26.78.254:46745)
0929 19:05:31.636385 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.636387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.636598 (+ 211us) server_negotiation.cc:407] Connection header received
0929 19:05:31.637053 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.637054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.637059 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.637124 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.638410 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.638411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.638411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.638415 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.638427 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.638430 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.638476 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.638576 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.638589 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:05:31.638797 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.636276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46745 (local address 127.26.78.194:60497)
0929 19:05:31.636465 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.636508 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.636970 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.637221 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.637224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.637232 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.638280 (+ 1048us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.638281 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.638309 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.638337 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.638489 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.638492 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.638494 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.638548 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.638598 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260929 19:05:31.640039 31883 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46745
I20260929 19:05:31.640146 31883 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:31.640223 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 31755
I20260929 19:05:31.640337 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:31.640331 31883 heartbeater.cc:511] Master 127.26.78.254:46745 requested a full tablet report, sending...
I20260929 19:05:31.640530 26939 external_mini_cluster.cc:1428] Setting key b084643bfce619d92218e291f1855bc4
I20260929 19:05:31.640738 31564 ts_manager.cc:194] Registered new tserver with Master: 8a3ac664729942a78f9d8bcb48c30851 (127.26.78.194:34543)
I20260929 19:05:31.641628 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46745
--builtin_ntp_servers=127.26.78.212:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:05:31.722637 31886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:31.722854 31886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:31.722877 31886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:31.723999 31886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:31.724035 31886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:31.724061 31886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:31.725332 31886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:46745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:31.725517 31886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:31.725863 31886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:05:31.731091 31886 server_base.cc:1034] running on GCE node
W20260929 19:05:31.731052 31892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.731060 31894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.731065 31891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.731472 31886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:31.731671 31886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:31.732815 31886 hybrid_clock.cc:648] HybridClock initialized: now 1790708731732798 us; error 33 us; skew 500 ppm
I20260929 19:05:31.734007 31886 webserver.cc:460] Webserver started at http://127.26.78.195:35247/ using document root <none> and password file <none>
I20260929 19:05:31.734197 31886 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:31.734239 31886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:31.734329 31886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:31.735325 31886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "5f777f94268348818208d7ca24f8878a"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "f779ffa8c3db1f75e6d7830e631f60e1"
server_key_iv: "bf15cd4a09fd9c5529275246832cc5c1"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.735617 31886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "5f777f94268348818208d7ca24f8878a"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "f779ffa8c3db1f75e6d7830e631f60e1"
server_key_iv: "bf15cd4a09fd9c5529275246832cc5c1"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.737003 31886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:31.737701 31900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.737885 31886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:31.737968 31886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "5f777f94268348818208d7ca24f8878a"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "f779ffa8c3db1f75e6d7830e631f60e1"
server_key_iv: "bf15cd4a09fd9c5529275246832cc5c1"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.738023 31886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:31.741221 31886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:31.741490 31886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:31.741633 31886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:31.741906 31886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:31.742226 31886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:31.742260 31886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.742326 31886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:31.742353 31886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.747792 31886 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:35853
I20260929 19:05:31.747941 32013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:35853 every 8 connection(s)
I20260929 19:05:31.748217 31886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:05:31.751276 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.748960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:60587 (local address 127.26.78.254:46745)
0929 19:05:31.749002 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.749004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.749244 (+ 240us) server_negotiation.cc:407] Connection header received
0929 19:05:31.749776 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.749777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.749785 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.749891 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.751087 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.751087 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.751088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.751091 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.751101 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.751104 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.751148 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.751206 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.751220 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:05:31.751494 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.748920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:46745 (local address 127.26.78.195:60587)
0929 19:05:31.749115 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.749149 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.749659 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.749969 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.749973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.749981 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.750948 (+ 967us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.750948 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.750977 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.751004 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.751147 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.751149 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.751152 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.751201 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.751251 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260929 19:05:31.752015 32014 heartbeater.cc:347] Connected to a master server at 127.26.78.254:46745
I20260929 19:05:31.752223 32014 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:31.752377 32014 heartbeater.cc:511] Master 127.26.78.254:46745 requested a full tablet report, sending...
I20260929 19:05:31.752734 31563 ts_manager.cc:194] Registered new tserver with Master: 5f777f94268348818208d7ca24f8878a (127.26.78.195:35853)
I20260929 19:05:31.756168 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 31886
I20260929 19:05:31.756249 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:31.756466 26939 external_mini_cluster.cc:1428] Setting key dd53d582e9f1355fccfda92449354acb
I20260929 19:05:31.757330 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:05:31.759801 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.758786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.26.78.254:46745)
0929 19:05:31.758853 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.758859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.758983 (+ 124us) server_negotiation.cc:407] Connection header received
0929 19:05:31.759042 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.759043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.759047 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.759100 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.759619 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.759620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.759621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.759624 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.759639 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.759643 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.759682 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.759728 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.759741 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:05:31.761459 31563 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:31.763156 31563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:31.766896 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.765277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:35853 (local address 127.0.0.1:49610)
0929 19:05:31.765636 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.765645 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.765680 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.765906 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.765913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.765919 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.766435 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.766435 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.766452 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.766477 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.766739 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.766745 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.766747 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.766810 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.766825 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":4,"thread_start_us":100,"threads_started":1}
I20260929 19:05:31.766952 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.765465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.26.78.195:35853)
0929 19:05:31.765730 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.765734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.765744 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:31.765769 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.765771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.765778 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.765906 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.766576 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.766583 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.766583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.766588 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.766624 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.766649 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.766712 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.766849 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.766865 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260929 19:05:31.767611 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.765276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:38521 (local address 127.0.0.1:52400)
0929 19:05:31.765410 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.765453 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.765511 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.765905 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.765913 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.765919 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.767200 (+ 1281us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.767200 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.767206 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.767216 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.767473 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.767474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.767475 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.767530 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.767544 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260929 19:05:31.767644 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.765535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.26.78.193:38521)
0929 19:05:31.765716 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.765721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.765733 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:05:31.765761 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.765763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.765778 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.765896 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.767312 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.767322 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.767323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.767327 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.767360 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.767386 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.767447 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.767544 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.767561 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260929 19:05:31.767642 31948 tablet_service.cc:1467] Processing CreateTablet for tablet a11aec48ef2b49cf89f786fa31ab6dde (DEFAULT_TABLE table=test-table [id=df4a6ce7a96f445a8a289c84aeda912d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:31.767913 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.765282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:34543 (local address 127.0.0.1:48338)
0929 19:05:31.765601 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.765611 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.765646 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.766687 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.766688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.766713 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.767322 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.767323 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.767331 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.767343 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.767766 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.767767 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.767768 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.767839 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.767853 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":81,"threads_started":1}
I20260929 19:05:31.768034 31948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11aec48ef2b49cf89f786fa31ab6dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.768119 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.765535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48338 (local address 127.26.78.194:34543)
0929 19:05:31.766262 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.766265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.766273 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:31.766294 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.766296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.766304 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.766442 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.767411 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.767417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.767417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.767422 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.767453 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.767473 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.767530 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.768017 (+ 487us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.768034 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":75,"threads_started":1}
I20260929 19:05:31.768651 31817 tablet_service.cc:1467] Processing CreateTablet for tablet a11aec48ef2b49cf89f786fa31ab6dde (DEFAULT_TABLE table=test-table [id=df4a6ce7a96f445a8a289c84aeda912d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:31.768975 31817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11aec48ef2b49cf89f786fa31ab6dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.770459 31684 tablet_service.cc:1467] Processing CreateTablet for tablet a11aec48ef2b49cf89f786fa31ab6dde (DEFAULT_TABLE table=test-table [id=df4a6ce7a96f445a8a289c84aeda912d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:31.770789 31684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11aec48ef2b49cf89f786fa31ab6dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.771834 32029 tablet_bootstrap.cc:492] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851: Bootstrap starting.
I20260929 19:05:31.771920 32028 tablet_bootstrap.cc:492] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Bootstrap starting.
I20260929 19:05:31.772508 32029 tablet_bootstrap.cc:654] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.772590 32028 tablet_bootstrap.cc:654] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.772830 32029 log.cc:824] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.772873 32028 log.cc:824] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.773677 32028 tablet_bootstrap.cc:492] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: No bootstrap required, opened a new log
I20260929 19:05:31.773779 32028 ts_tablet_manager.cc:1397] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:31.774056 32032 tablet_bootstrap.cc:492] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f: Bootstrap starting.
I20260929 19:05:31.774602 32032 tablet_bootstrap.cc:654] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.774946 32032 log.cc:824] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.775720 32032 tablet_bootstrap.cc:492] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f: No bootstrap required, opened a new log
I20260929 19:05:31.775805 32032 ts_tablet_manager.cc:1397] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:31.776072 32028 raft_consensus.cc:348] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.776202 32028 raft_consensus.cc:374] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.776252 32028 raft_consensus.cc:729] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f777f94268348818208d7ca24f8878a, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.776382 32028 consensus_queue.cc:260] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.776644 32028 ts_tablet_manager.cc:1428] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:05:31.776698 32014 heartbeater.cc:503] Master 127.26.78.254:46745 was elected leader, sending a full tablet report...
I20260929 19:05:31.777103 32029 tablet_bootstrap.cc:492] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851: No bootstrap required, opened a new log
I20260929 19:05:31.777179 32029 ts_tablet_manager.cc:1397] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260929 19:05:31.778679 32032 raft_consensus.cc:348] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.778899 32032 raft_consensus.cc:374] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.778945 32032 raft_consensus.cc:729] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56079e211ede49c3861fd3781358755f, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.779091 32032 consensus_queue.cc:260] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.779368 32032 ts_tablet_manager.cc:1428] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260929 19:05:31.779476 31752 heartbeater.cc:503] Master 127.26.78.254:46745 was elected leader, sending a full tablet report...
I20260929 19:05:31.779946 32029 raft_consensus.cc:348] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.780118 32029 raft_consensus.cc:374] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.780164 32029 raft_consensus.cc:729] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a3ac664729942a78f9d8bcb48c30851, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.780278 32029 consensus_queue.cc:260] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.780542 32029 ts_tablet_manager.cc:1428] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260929 19:05:31.780570 31883 heartbeater.cc:503] Master 127.26.78.254:46745 was elected leader, sending a full tablet report...
I20260929 19:05:31.782337 32035 raft_consensus.cc:482] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:31.782413 32035 raft_consensus.cc:504] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.782680 32035 leader_election.cc:288] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f777f94268348818208d7ca24f8878a (127.26.78.195:35853), 8a3ac664729942a78f9d8bcb48c30851 (127.26.78.194:34543)
I20260929 19:05:31.784106 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.783037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:49931 (local address 127.26.78.195:35853)
0929 19:05:31.783114 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.783116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.783124 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:31.783146 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.783147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.783152 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.783203 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.783871 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.783873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.783873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.783878 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.783893 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.783898 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.783973 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.784027 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.784041 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 19:05:31.784104 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.782821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:35853 (local address 127.26.78.193:49931)
0929 19:05:31.782861 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.782873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.782910 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.783258 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.783259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.783261 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.783761 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.783762 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.783769 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.783780 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.783968 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.783969 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.783969 (+ 0us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.784026 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.784041 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 19:05:31.784173 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.783058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:32931 (local address 127.26.78.194:34543)
0929 19:05:31.783106 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.783108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.783116 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:31.783137 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.783138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.783143 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.783193 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.783952 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.783953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.783954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.783958 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.783971 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.783975 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.784026 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.784094 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.784108 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:31.784207 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.782866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:34543 (local address 127.26.78.193:32931)
0929 19:05:31.783004 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.783016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:31.783062 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:31.783213 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:31.783214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:31.783217 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:31.783847 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:05:31.783847 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:31.783855 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:05:31.783867 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:31.784038 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:31.784039 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:31.784040 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:05:31.784093 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:05:31.784107 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260929 19:05:31.784513 31837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11aec48ef2b49cf89f786fa31ab6dde" candidate_uuid: "56079e211ede49c3861fd3781358755f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a3ac664729942a78f9d8bcb48c30851" is_pre_election: true
I20260929 19:05:31.784608 31837 raft_consensus.cc:2454] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56079e211ede49c3861fd3781358755f in term 0.
I20260929 19:05:31.784622 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11aec48ef2b49cf89f786fa31ab6dde" candidate_uuid: "56079e211ede49c3861fd3781358755f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f777f94268348818208d7ca24f8878a" is_pre_election: true
I20260929 19:05:31.784750 31968 raft_consensus.cc:2454] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56079e211ede49c3861fd3781358755f in term 0.
I20260929 19:05:31.784816 31639 leader_election.cc:302] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56079e211ede49c3861fd3781358755f, 8a3ac664729942a78f9d8bcb48c30851; no voters:
I20260929 19:05:31.784914 32035 raft_consensus.cc:2781] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:31.784981 32035 raft_consensus.cc:482] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:31.785017 32035 raft_consensus.cc:3037] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.785742 32035 raft_consensus.cc:504] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.785931 32035 leader_election.cc:288] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [CANDIDATE]: Term 1 election: Requested vote from peers 5f777f94268348818208d7ca24f8878a (127.26.78.195:35853), 8a3ac664729942a78f9d8bcb48c30851 (127.26.78.194:34543)
I20260929 19:05:31.786098 31837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11aec48ef2b49cf89f786fa31ab6dde" candidate_uuid: "56079e211ede49c3861fd3781358755f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a3ac664729942a78f9d8bcb48c30851"
I20260929 19:05:31.786185 31837 raft_consensus.cc:3037] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.786824 31837 raft_consensus.cc:2454] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56079e211ede49c3861fd3781358755f in term 1.
I20260929 19:05:31.786957 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11aec48ef2b49cf89f786fa31ab6dde" candidate_uuid: "56079e211ede49c3861fd3781358755f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f777f94268348818208d7ca24f8878a"
I20260929 19:05:31.787009 31639 leader_election.cc:302] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56079e211ede49c3861fd3781358755f, 8a3ac664729942a78f9d8bcb48c30851; no voters:
I20260929 19:05:31.787051 31968 raft_consensus.cc:3037] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.787703 31968 raft_consensus.cc:2454] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56079e211ede49c3861fd3781358755f in term 1.
I20260929 19:05:31.787827 32035 raft_consensus.cc:2781] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:31.787979 32035 raft_consensus.cc:686] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [term 1 LEADER]: Becoming Leader. State: Replica: 56079e211ede49c3861fd3781358755f, State: Running, Role: LEADER
I20260929 19:05:31.788089 32035 consensus_queue.cc:237] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } }
I20260929 19:05:31.788836 31563 catalog_manager.cc:5495] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f reported cstate change: term changed from 0 to 1, leader changed from <none> to 56079e211ede49c3861fd3781358755f (127.26.78.193). New cstate: current_term: 1 leader_uuid: "56079e211ede49c3861fd3781358755f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56079e211ede49c3861fd3781358755f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 38521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 } health_report { overall_health: UNKNOWN } } }
I20260929 19:05:31.797640 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.796747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.26.78.193:38521)
0929 19:05:31.796826 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.796828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.796841 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:05:31.796869 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.796869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.796881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.796930 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.797493 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.797494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.797494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.797497 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.797508 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.797511 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.797551 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.797582 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.797592 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 19:05:31.799587 31837 raft_consensus.cc:1261] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 1 FOLLOWER]: Refusing update from remote peer 56079e211ede49c3861fd3781358755f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:31.799904 31968 raft_consensus.cc:1261] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 1 FOLLOWER]: Refusing update from remote peer 56079e211ede49c3861fd3781358755f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:31.799944 32038 consensus_queue.cc:1035] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a3ac664729942a78f9d8bcb48c30851" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:31.800141 32038 consensus_queue.cc:1035] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f777f94268348818208d7ca24f8878a" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:31.802345 32041 mvcc.cc:205] Tried to move back new op lower bound from 7334742965448171520 to 7334742965404585984. Current Snapshot: MvccSnapshot[applied={T|T < 7334742965448171520}]
I20260929 19:05:31.803277 32043 mvcc.cc:205] Tried to move back new op lower bound from 7334742965448171520 to 7334742965404585984. Current Snapshot: MvccSnapshot[applied={T|T < 7334742965448171520 or (T in {7334742965448171520})}]
I20260929 19:05:31.803463 32042 mvcc.cc:205] Tried to move back new op lower bound from 7334742965448171520 to 7334742965404585984. Current Snapshot: MvccSnapshot[applied={T|T < 7334742965448171520}]
I20260929 19:05:31.804864 31564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:31.804940 31564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:31.805639 31564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 941064e096ae444fa78620b0bf8054ad: Sending DeleteTablet for 3 replicas of tablet a11aec48ef2b49cf89f786fa31ab6dde
I20260929 19:05:31.805979 31684 tablet_service.cc:1514] Processing DeleteTablet for tablet a11aec48ef2b49cf89f786fa31ab6dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:31 UTC) from {username='slave'} at 127.0.0.1:52400
I20260929 19:05:31.806011 31948 tablet_service.cc:1514] Processing DeleteTablet for tablet a11aec48ef2b49cf89f786fa31ab6dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:31 UTC) from {username='slave'} at 127.0.0.1:49610
I20260929 19:05:31.806260 32050 tablet_replica.cc:330] T a11aec48ef2b49cf89f786fa31ab6dde P 56079e211ede49c3861fd3781358755f: stopping tablet replica
I20260929 19:05:31.806252 31817 tablet_service.cc:1514] Processing DeleteTablet for tablet a11aec48ef2b49cf89f786fa31ab6dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:31 UTC) from {username='slave'} at 127.0.0.1:48338
I20260929 19:05:31.806416 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 31624
I20260929 19:05:31.807066 32051 tablet_replica.cc:330] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: stopping tablet replica
I20260929 19:05:31.807175 32051 raft_consensus.cc:2229] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:31.807302 32051 raft_consensus.cc:2258] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:31.807765 32051 ts_tablet_manager.cc:1905] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:31.809633 32051 ts_tablet_manager.cc:1918] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:31.809765 32051 log.cc:1196] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/a11aec48ef2b49cf89f786fa31ab6dde
I20260929 19:05:31.810139 32051 ts_tablet_manager.cc:1939] T a11aec48ef2b49cf89f786fa31ab6dde P 5f777f94268348818208d7ca24f8878a: Deleting consensus metadata
I20260929 19:05:31.810599 31549 catalog_manager.cc:4841] TS 5f777f94268348818208d7ca24f8878a (127.26.78.195:35853): tablet a11aec48ef2b49cf89f786fa31ab6dde (table test-table [id=df4a6ce7a96f445a8a289c84aeda912d]) successfully deleted
W20260929 19:05:31.813998 31551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:38521 (error 108)
I20260929 19:05:31.814114 32052 tablet_replica.cc:330] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851: stopping tablet replica
I20260929 19:05:31.814245 32052 raft_consensus.cc:2229] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:31.814370 32052 raft_consensus.cc:2258] T a11aec48ef2b49cf89f786fa31ab6dde P 8a3ac664729942a78f9d8bcb48c30851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:31.814635 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.814482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:38521 (local address 127.0.0.1:52416)
0929 19:05:31.814535 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.814592 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 19:05:31.814828 31551 catalog_manager.cc:4584] TS 56079e211ede49c3861fd3781358755f (127.26.78.193:38521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a11aec48ef2b49cf89f786fa31ab6dde: Network error: Client connection negotiation failed: client connection to 127.26.78.193:38521: connect: Connection refused (error 111)
I20260929 19:05:31.814872 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 31755
I20260929 19:05:31.820181 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.820026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:34543 (local address 127.0.0.1:48354)
0929 19:05:31.820097 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:31.820132 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.194:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 19:05:31.820384 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 31886
I20260929 19:05:31.825029 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 31533
2026-09-29T19:05:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (578 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 19:05:31.845573 26939 mini_oidc.cc:152] Starting JWKS server
I20260929 19:05:31.846774 26939 webserver.cc:460] Webserver started at https://127.0.0.1:39569/ using document root <none> and password file <none>
I20260929 19:05:31.846858 26939 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 19:05:31.846985 26939 webserver.cc:460] Webserver started at http://127.0.0.1:35995/ using document root <none> and password file <none>
2026-09-29T19:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:31Z Disabled control of system clock
I20260929 19:05:31.855149 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43061
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:45415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:43061
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39569/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/testchainca.cert with env {}
W20260929 19:05:31.938494 32059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:31.938716 32059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:31.938741 32059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:31.939960 32059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:31.940008 32059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:31.940023 32059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:31.940038 32059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:31.940052 32059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:05:31.940068 32059 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:05:31.940080 32059 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39569/jwks.json
I20260929 19:05:31.942015 32059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:43061
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39569/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:31.942399 32059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:31.942602 32059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:31.947634 32065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:31.947670 32064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.947705 32059 server_base.cc:1034] running on GCE node
W20260929 19:05:31.947631 32067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:31.948146 32059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:31.948460 32059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:31.949642 32059 hybrid_clock.cc:648] HybridClock initialized: now 1790708731949635 us; error 38 us; skew 500 ppm
I20260929 19:05:31.950989 32059 webserver.cc:460] Webserver started at http://127.26.78.254:33393/ using document root <none> and password file <none>
I20260929 19:05:31.951241 32059 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:31.951331 32059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:31.951506 32059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:31.952612 32059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "9158abd48bf44830be3871f847a52018"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "b57ee81b0924d2e1bb05e98c12066f50"
server_key_iv: "d167d54184d1a1d2bbdbd8c76fb6fb46"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.952996 32059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "9158abd48bf44830be3871f847a52018"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "b57ee81b0924d2e1bb05e98c12066f50"
server_key_iv: "d167d54184d1a1d2bbdbd8c76fb6fb46"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.954401 32059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:31.955132 32073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:31.955314 32059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:31.955415 32059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "9158abd48bf44830be3871f847a52018"
format_stamp: "Formatted at 2026-09-29 19:05:31 on dist-test-slave-xz51"
server_key: "b57ee81b0924d2e1bb05e98c12066f50"
server_key_iv: "d167d54184d1a1d2bbdbd8c76fb6fb46"
server_key_version: "encryptionkey@0"
I20260929 19:05:31.955497 32059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:31.964860 32059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:31.965173 32059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:31.965332 32059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:31.969120 32059 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:43061
I20260929 19:05:31.969187 32125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:43061 every 8 connection(s)
I20260929 19:05:31.969514 32059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:31.970495 32126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:31.972782 32126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018: Bootstrap starting.
I20260929 19:05:31.973397 32126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:31.973699 32126 log.cc:824] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:31.974424 32126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018: No bootstrap required, opened a new log
I20260929 19:05:31.976022 32126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9158abd48bf44830be3871f847a52018" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43061 } }
I20260929 19:05:31.976161 32126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:31.976213 32126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9158abd48bf44830be3871f847a52018, State: Initialized, Role: FOLLOWER
I20260929 19:05:31.976364 32126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9158abd48bf44830be3871f847a52018" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43061 } }
I20260929 19:05:31.976425 32126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:31.976446 32126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:31.976518 32126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:31.977052 32126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9158abd48bf44830be3871f847a52018" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43061 } }
I20260929 19:05:31.977167 32126 leader_election.cc:302] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9158abd48bf44830be3871f847a52018; no voters:
I20260929 19:05:31.977350 32126 leader_election.cc:288] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:31.977424 32129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:31.977569 32129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [term 1 LEADER]: Becoming Leader. State: Replica: 9158abd48bf44830be3871f847a52018, State: Running, Role: LEADER
I20260929 19:05:31.977605 32126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:31.977669 32129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9158abd48bf44830be3871f847a52018" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43061 } }
I20260929 19:05:31.978089 32131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9158abd48bf44830be3871f847a52018. Latest consensus state: current_term: 1 leader_uuid: "9158abd48bf44830be3871f847a52018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9158abd48bf44830be3871f847a52018" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43061 } } }
I20260929 19:05:31.978183 32131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:31.978283 32130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9158abd48bf44830be3871f847a52018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9158abd48bf44830be3871f847a52018" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43061 } } }
I20260929 19:05:31.978370 32130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:31.978825 32137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:31.979383 32137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:31.979483 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 32059
I20260929 19:05:31.979583 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:31.979750 26939 external_mini_cluster.cc:1428] Setting key 9f54c231230ef8cb912fc3a6382c457a
I20260929 19:05:31.981021 32137 catalog_manager.cc:1347] Generated new cluster ID: 946d0fcfb2f640c8ab067eb1a8945ba0
I20260929 19:05:31.981073 32137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:31.987041 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:31.980169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.26.78.254:43061)
0929 19:05:31.980607 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:31.980615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:31.980639 (+ 24us) server_negotiation.cc:407] Connection header received
0929 19:05:31.980774 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:31.980781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:31.980861 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:31.982140 (+ 1279us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:31.982236 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.983640 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.984742 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:31.985415 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:31.985548 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:31.986339 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:31.986344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:31.986344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:31.986553 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:31.986594 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:31.986608 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:31.986679 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:31.986747 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 19:05:31.986831 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":64,"threads_started":1}
I20260929 19:05:31.988013 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:05:31.991989 32137 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:31.992491 32137 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:31.999136 32137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9158abd48bf44830be3871f847a52018: Generated new TSK 0
I20260929 19:05:31.999357 32137 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 19:05:32.021550 32156 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 19:05:32.021661 32156 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260929 19:05:32.022207 32155 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:32.007013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43061 (local address 127.0.0.1:46146)
0929 19:05:32.007230 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:32.007240 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:32.007319 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:32.007523 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:32.007524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:32.007526 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:05:32.007604 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.007606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.008269 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.008270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.008879 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.008881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.009259 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.009260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.009322 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.009346 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:05:32.021831 (+ 12485us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 19:05:32.021956 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.78.254:43061: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260929 19:05:32.022560 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:32.007086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.26.78.254:43061)
0929 19:05:32.007154 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:32.007156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:32.007310 (+ 154us) server_negotiation.cc:407] Connection header received
0929 19:05:32.007423 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:32.007424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:32.007434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:32.007500 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:05:32.007680 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.008182 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.008977 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.009165 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.009243 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.009475 (+ 232us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:05:32.021678 (+ 12203us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 19:05:32.022469 (+ 791us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46146: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260929 19:05:32.022615 32148 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46146: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.26.78.254:43061: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 19:05:32.022856 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 32059
2026-09-29T19:05:32Z chronyd exiting
I20260929 19:05:32.058439 26939 test_util.cc:175] -----------------------------------------------
I20260929 19:05:32.058524 26939 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790708710206825-26939-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 19:05:32.059367 26939 mini_oidc.cc:152] Starting JWKS server
I20260929 19:05:32.060014 26939 webserver.cc:460] Webserver started at https://127.0.0.1:38003/ using document root <none> and password file <none>
I20260929 19:05:32.060092 26939 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 19:05:32.060241 26939 webserver.cc:460] Webserver started at http://127.0.0.1:33369/ using document root <none> and password file <none>
2026-09-29T19:05:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:32Z Disabled control of system clock
I20260929 19:05:32.068033 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:34929
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:34589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:34929
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38003/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/testchainca.cert with env {}
W20260929 19:05:32.148635 32162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:32.148854 32162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:32.148874 32162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:32.150120 32162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:32.150162 32162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:32.150174 32162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:32.150187 32162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:32.150256 32162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:05:32.150274 32162 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:05:32.150309 32162 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38003/jwks.json
I20260929 19:05:32.151891 32162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:34929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:34929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38003/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:32.152163 32162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:32.152398 32162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:32.157444 32170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:32.157493 32162 server_base.cc:1034] running on GCE node
W20260929 19:05:32.157451 32167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:32.157451 32168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:32.157944 32162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:32.158217 32162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:32.159410 32162 hybrid_clock.cc:648] HybridClock initialized: now 1790708732159390 us; error 35 us; skew 500 ppm
I20260929 19:05:32.160949 32162 webserver.cc:460] Webserver started at http://127.26.78.254:37273/ using document root <none> and password file <none>
I20260929 19:05:32.161221 32162 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:32.161269 32162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:32.161437 32162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:32.162587 32162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "cb36f8c0264543c08de140cf97506db8"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "85ae266909d8d68c3d0a75f3e971a25b"
server_key_iv: "a979c215d1b0ae61924b4d8a13aa78c7"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.162967 32162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "cb36f8c0264543c08de140cf97506db8"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "85ae266909d8d68c3d0a75f3e971a25b"
server_key_iv: "a979c215d1b0ae61924b4d8a13aa78c7"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.164280 32162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:32.165011 32176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:32.165294 32162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:32.165400 32162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "cb36f8c0264543c08de140cf97506db8"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "85ae266909d8d68c3d0a75f3e971a25b"
server_key_iv: "a979c215d1b0ae61924b4d8a13aa78c7"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.165478 32162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:32.184471 32162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:32.184828 32162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:32.184994 32162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:32.188951 32162 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:34929
I20260929 19:05:32.188992 32228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:34929 every 8 connection(s)
I20260929 19:05:32.189419 32162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:32.190199 32229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:32.192556 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 32162
I20260929 19:05:32.192680 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:32.192850 26939 external_mini_cluster.cc:1428] Setting key af840c4323f2fca617205fd9c35b8871
I20260929 19:05:32.193163 32229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8: Bootstrap starting.
I20260929 19:05:32.193809 32229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:32.194126 32229 log.cc:824] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:32.194823 32229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8: No bootstrap required, opened a new log
I20260929 19:05:32.196521 32229 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb36f8c0264543c08de140cf97506db8" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34929 } }
I20260929 19:05:32.196691 32229 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:32.196738 32229 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb36f8c0264543c08de140cf97506db8, State: Initialized, Role: FOLLOWER
I20260929 19:05:32.196888 32229 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb36f8c0264543c08de140cf97506db8" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34929 } }
I20260929 19:05:32.196985 32229 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:32.197024 32229 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:32.197077 32229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:32.197608 32229 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb36f8c0264543c08de140cf97506db8" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34929 } }
I20260929 19:05:32.197732 32229 leader_election.cc:302] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb36f8c0264543c08de140cf97506db8; no voters:
I20260929 19:05:32.197950 32229 leader_election.cc:288] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:32.197984 32234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:32.198148 32234 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [term 1 LEADER]: Becoming Leader. State: Replica: cb36f8c0264543c08de140cf97506db8, State: Running, Role: LEADER
I20260929 19:05:32.198220 32229 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:32.198228 32234 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb36f8c0264543c08de140cf97506db8" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34929 } }
I20260929 19:05:32.198589 32234 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb36f8c0264543c08de140cf97506db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb36f8c0264543c08de140cf97506db8" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34929 } } }
I20260929 19:05:32.198683 32234 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:32.198832 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:32.193383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.26.78.254:34929)
0929 19:05:32.193604 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:32.193613 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:32.193638 (+ 25us) server_negotiation.cc:407] Connection header received
0929 19:05:32.194204 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:32.194211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:32.194316 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:32.194476 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:32.194671 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.195627 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.196458 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.196952 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.197098 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.197848 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:32.197851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:32.197852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:32.198100 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:32.198145 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:32.198160 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:32.198230 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:32.198466 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 19:05:32.198558 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260929 19:05:32.198850 32234 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb36f8c0264543c08de140cf97506db8. Latest consensus state: current_term: 1 leader_uuid: "cb36f8c0264543c08de140cf97506db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb36f8c0264543c08de140cf97506db8" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34929 } } }
I20260929 19:05:32.198936 32234 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:32.199040 32241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:32.199687 32241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:32.199824 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:05:32.201319 32241 catalog_manager.cc:1347] Generated new cluster ID: e705f561e9da4ffe8703a76b0f290d08
I20260929 19:05:32.201371 32241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:32.221915 32241 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:32.222484 32241 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:32.227820 32241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb36f8c0264543c08de140cf97506db8: Generated new TSK 0
I20260929 19:05:32.228039 32241 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 19:05:32.254528 32259 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 19:05:32.254629 32259 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 19:05:32.254957 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:32.240327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.26.78.254:34929)
0929 19:05:32.240374 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:32.240376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:32.240491 (+ 115us) server_negotiation.cc:407] Connection header received
0929 19:05:32.240539 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:32.240540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:32.240553 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:32.240639 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:05:32.240762 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.241594 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.242427 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.242745 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.242820 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.242961 (+ 141us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:05:32.254719 (+ 11758us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 19:05:32.254880 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56362: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 19:05:32.255028 32232 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56362: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260929 19:05:32.255048 32258 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:32.240253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:34929 (local address 127.0.0.1:56362)
0929 19:05:32.240410 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:32.240421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:32.240484 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:32.240643 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:32.240643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:32.240645 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:05:32.240697 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.240698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.241694 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.241695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.242346 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.242348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.242831 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.242831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.242872 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.242877 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:05:32.254846 (+ 11969us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 19:05:32.254945 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.78.254:34929: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260929 19:05:32.255381 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 32162
2026-09-29T19:05:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (233 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 19:05:32.293143 26939 mini_oidc.cc:152] Starting JWKS server
I20260929 19:05:32.293927 26939 webserver.cc:460] Webserver started at https://127.0.0.1:40123/ using document root <none> and password file <none>
I20260929 19:05:32.293999 26939 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 19:05:32.294163 26939 webserver.cc:460] Webserver started at http://127.0.0.1:36515/ using document root <none> and password file <none>
2026-09-29T19:05:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:32Z Disabled control of system clock
I20260929 19:05:32.302002 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33049
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:33049
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40123/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 19:05:32.381767 32265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:32.381995 32265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:32.382017 32265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:32.383205 32265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:32.383249 32265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:32.383262 32265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:32.383323 32265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:32.383350 32265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:05:32.383373 32265 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:05:32.383399 32265 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40123/jwks.json
I20260929 19:05:32.384778 32265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:33049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40123/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:32.385048 32265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:32.385352 32265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:32.390156 32270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:32.390163 32271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:32.390164 32273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:32.390473 32265 server_base.cc:1034] running on GCE node
I20260929 19:05:32.390704 32265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:32.391060 32265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:32.392216 32265 hybrid_clock.cc:648] HybridClock initialized: now 1790708732392203 us; error 49 us; skew 500 ppm
I20260929 19:05:32.393474 32265 webserver.cc:460] Webserver started at http://127.26.78.254:44999/ using document root <none> and password file <none>
I20260929 19:05:32.393733 32265 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:32.393805 32265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:32.393983 32265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:32.395076 32265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "39bfa647d886450fa992eda9ee486c67"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "65fbdc67578a338d6069d4cc43fabd6f"
server_key_iv: "9ab3a652983fa766ddca6d33edf53ae1"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.395458 32265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "39bfa647d886450fa992eda9ee486c67"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "65fbdc67578a338d6069d4cc43fabd6f"
server_key_iv: "9ab3a652983fa766ddca6d33edf53ae1"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.396733 32265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:32.397436 32279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:32.397627 32265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:32.397724 32265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "39bfa647d886450fa992eda9ee486c67"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "65fbdc67578a338d6069d4cc43fabd6f"
server_key_iv: "9ab3a652983fa766ddca6d33edf53ae1"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.397814 32265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:32.414314 32265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:32.414654 32265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:32.414816 32265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:32.419035 32265 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:33049
I20260929 19:05:32.419070 32331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:33049 every 8 connection(s)
I20260929 19:05:32.419437 32265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:32.420095 32332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:32.423099 32332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67: Bootstrap starting.
I20260929 19:05:32.423709 32332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:32.424001 32332 log.cc:824] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:32.424773 32332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67: No bootstrap required, opened a new log
I20260929 19:05:32.426322 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 32265
I20260929 19:05:32.426433 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:32.426620 26939 external_mini_cluster.cc:1428] Setting key 4fd1f64d7da019a74a43fee669d09745
I20260929 19:05:32.426510 32332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bfa647d886450fa992eda9ee486c67" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33049 } }
I20260929 19:05:32.426666 32332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:32.426729 32332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39bfa647d886450fa992eda9ee486c67, State: Initialized, Role: FOLLOWER
I20260929 19:05:32.426874 32332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bfa647d886450fa992eda9ee486c67" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33049 } }
I20260929 19:05:32.426928 32332 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:32.426949 32332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:32.426977 32332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:32.427719 32332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bfa647d886450fa992eda9ee486c67" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33049 } }
I20260929 19:05:32.427830 32332 leader_election.cc:302] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39bfa647d886450fa992eda9ee486c67; no voters:
I20260929 19:05:32.427989 32332 leader_election.cc:288] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:32.428045 32337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:32.428186 32337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [term 1 LEADER]: Becoming Leader. State: Replica: 39bfa647d886450fa992eda9ee486c67, State: Running, Role: LEADER
I20260929 19:05:32.428333 32332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:32.428321 32337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bfa647d886450fa992eda9ee486c67" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33049 } }
I20260929 19:05:32.428892 32338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39bfa647d886450fa992eda9ee486c67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bfa647d886450fa992eda9ee486c67" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33049 } } }
I20260929 19:05:32.428885 32339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39bfa647d886450fa992eda9ee486c67. Latest consensus state: current_term: 1 leader_uuid: "39bfa647d886450fa992eda9ee486c67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bfa647d886450fa992eda9ee486c67" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33049 } } }
I20260929 19:05:32.429040 32338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:32.429052 32339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67 [sys.catalog]: This master's current role is: LEADER
W20260929 19:05:32.430336 32353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:05:32.430398 32353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:05:32.430444 32346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:32.430676 32346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:32.432256 32346 catalog_manager.cc:1347] Generated new cluster ID: a5ab2c034d2448a5b73e3987d761e4ed
I20260929 19:05:32.432322 32346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:32.433413 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:32.427166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.26.78.254:33049)
0929 19:05:32.427366 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:32.427375 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:32.427409 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:05:32.427540 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:32.427548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:32.427636 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:32.427814 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:32.428085 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.430052 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.431183 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.431794 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.431924 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.432714 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:32.432718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:32.432719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:32.432923 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:32.432964 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:32.432979 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:32.433049 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:32.433122 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 19:05:32.433199 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260929 19:05:32.434362 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:05:32.443153 32346 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:32.443676 32346 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:32.446050 32346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39bfa647d886450fa992eda9ee486c67: Generated new TSK 0
I20260929 19:05:32.446223 32346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:05:32.461009 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:32.457278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52200 (local address 127.26.78.254:33049)
0929 19:05:32.457344 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:32.457346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:32.457439 (+ 93us) server_negotiation.cc:407] Connection header received
0929 19:05:32.457516 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:32.457517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:32.457527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:32.457589 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:05:32.457736 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.458284 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.459431 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.459629 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.459681 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.459871 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:05:32.460805 (+ 934us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 19:05:32.460938 (+ 133us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52200: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260929 19:05:32.461110 32361 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:32.457208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33049 (local address 127.0.0.1:52200)
0929 19:05:32.457367 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:32.457379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:32.457445 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:32.457611 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:32.457612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:32.457615 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:05:32.457672 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.457674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.458378 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.458379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.459345 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.459348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.459708 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.459709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.459770 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.459776 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:05:32.460908 (+ 1132us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 19:05:32.461006 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.78.254:33049: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
W20260929 19:05:32.461346 32336 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52200: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260929 19:05:32.461431 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 32265
E20260929 19:05:32.472324 32362 webserver.cc:533] Webserver: sslize failed:
E20260929 19:05:32.472398 32362 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T19:05:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (205 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T19:05:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:32Z Disabled control of system clock
I20260929 19:05:32.506464 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:39655
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:39655
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/testchainca.cert with env {}
W20260929 19:05:32.585528 32366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:32.585738 32366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:32.585760 32366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:32.586853 32366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:32.586887 32366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:32.586900 32366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:32.586912 32366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:32.586925 32366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:05:32.586936 32366 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:05:32.586947 32366 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 19:05:32.588335 32366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:39655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:39655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:32.588519 32366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:32.588676 32366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:32.593531 32372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:32.593534 32371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:32.593596 32366 server_base.cc:1034] running on GCE node
W20260929 19:05:32.593503 32374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:32.593926 32366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:32.594219 32366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:32.595388 32366 hybrid_clock.cc:648] HybridClock initialized: now 1790708732595388 us; error 37 us; skew 500 ppm
I20260929 19:05:32.596653 32366 webserver.cc:460] Webserver started at http://127.26.78.254:38681/ using document root <none> and password file <none>
I20260929 19:05:32.596916 32366 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:32.596995 32366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:32.597146 32366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:32.598268 32366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "cb62d7a4fbd6450592e622899256cbd4"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "6eeda947aad19217301bb03ec59d62f1"
server_key_iv: "b730e391cc10b5bce526f40d267ec6c3"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.598626 32366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "cb62d7a4fbd6450592e622899256cbd4"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "6eeda947aad19217301bb03ec59d62f1"
server_key_iv: "b730e391cc10b5bce526f40d267ec6c3"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.599893 32366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:05:32.600630 32380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:32.600812 32366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:32.600919 32366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "cb62d7a4fbd6450592e622899256cbd4"
format_stamp: "Formatted at 2026-09-29 19:05:32 on dist-test-slave-xz51"
server_key: "6eeda947aad19217301bb03ec59d62f1"
server_key_iv: "b730e391cc10b5bce526f40d267ec6c3"
server_key_version: "encryptionkey@0"
I20260929 19:05:32.601003 32366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:32.611325 32366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:32.611642 32366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:32.611802 32366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:32.615690 32366 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:39655
I20260929 19:05:32.615723 32432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:39655 every 8 connection(s)
I20260929 19:05:32.616055 32366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:32.616971 32433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:32.619302 32433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4: Bootstrap starting.
I20260929 19:05:32.620024 32433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:32.620393 32433 log.cc:824] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:32.620723 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 32366
I20260929 19:05:32.620820 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:32.621007 26939 external_mini_cluster.cc:1428] Setting key 44c7836d80fbb83d1a319a14efb748db
I20260929 19:05:32.621130 32433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4: No bootstrap required, opened a new log
I20260929 19:05:32.623509 32433 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62d7a4fbd6450592e622899256cbd4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39655 } }
I20260929 19:05:32.623698 32433 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:32.623744 32433 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb62d7a4fbd6450592e622899256cbd4, State: Initialized, Role: FOLLOWER
I20260929 19:05:32.623872 32433 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62d7a4fbd6450592e622899256cbd4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39655 } }
I20260929 19:05:32.623958 32433 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:32.623994 32433 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:32.624046 32433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:32.624802 32433 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62d7a4fbd6450592e622899256cbd4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39655 } }
I20260929 19:05:32.624912 32433 leader_election.cc:302] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb62d7a4fbd6450592e622899256cbd4; no voters:
I20260929 19:05:32.625059 32433 leader_election.cc:288] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:32.625120 32438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:32.625288 32438 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [term 1 LEADER]: Becoming Leader. State: Replica: cb62d7a4fbd6450592e622899256cbd4, State: Running, Role: LEADER
I20260929 19:05:32.625313 32433 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:32.625375 32438 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62d7a4fbd6450592e622899256cbd4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39655 } }
I20260929 19:05:32.625696 32438 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb62d7a4fbd6450592e622899256cbd4. Latest consensus state: current_term: 1 leader_uuid: "cb62d7a4fbd6450592e622899256cbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62d7a4fbd6450592e622899256cbd4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39655 } } }
I20260929 19:05:32.625788 32438 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:32.625711 32439 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb62d7a4fbd6450592e622899256cbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62d7a4fbd6450592e622899256cbd4" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39655 } } }
I20260929 19:05:32.625924 32439 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4 [sys.catalog]: This master's current role is: LEADER
W20260929 19:05:32.627408 32453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:05:32.627502 32453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:05:32.627589 32454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:32.627753 32454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:32.628877 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:32.621622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59962 (local address 127.26.78.254:39655)
0929 19:05:32.621850 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:32.621860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:32.622820 (+ 960us) server_negotiation.cc:407] Connection header received
0929 19:05:32.623439 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:32.623447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:32.623509 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:32.623620 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:32.623899 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.625585 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.626646 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.627326 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.627469 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.628156 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:32.628159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:32.628160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:05:32.628348 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:32.628412 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:05:32.628427 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:32.628493 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:32.628614 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 19:05:32.628691 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260929 19:05:32.629487 32454 catalog_manager.cc:1347] Generated new cluster ID: 7791127057124dfc8ad3f41da1dc2a07
I20260929 19:05:32.629541 32454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:32.629897 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:05:32.641239 32454 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:32.641781 32454 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:32.645851 32454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb62d7a4fbd6450592e622899256cbd4: Generated new TSK 0
I20260929 19:05:32.646049 32454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:05:32.707315 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:32.653436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59978 (local address 127.26.78.254:39655)
0929 19:05:32.653483 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:32.653486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:32.653593 (+ 107us) server_negotiation.cc:407] Connection header received
0929 19:05:32.653664 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:32.653665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:32.653675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:32.653736 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:05:32.653899 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.654394 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.655565 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.655747 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.655798 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.655985 (+ 187us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:05:32.707092 (+ 51107us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 19:05:32.707243 (+ 151us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59978: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 19:05:32.707418 32437 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59978: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 19:05:32.707513 32462 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:32.653352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:39655 (local address 127.0.0.1:59978)
0929 19:05:32.653516 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:32.653528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:32.653597 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:32.653753 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:32.653754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:32.653755 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:05:32.653840 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.653843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.654485 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.654486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.655483 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:32.655486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:32.655827 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:32.655829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:32.655889 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:32.655895 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:05:32.707249 (+ 51354us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 19:05:32.707408 (+ 159us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.78.254:39655: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260929 19:05:32.707911 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 32366
2026-09-29T19:05:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (226 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:32 dist-test-slave-xz51 krb5kdc[32474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:32 dist-test-slave-xz51 krb5kdc[32474](info): set up 2 sockets
Sep 29 19:05:32 dist-test-slave-xz51 krb5kdc[32474](info): commencing operation
krb5kdc: starting...
W20260929 19:05:33.918183 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:05:34.037256 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38013
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:38013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:34.120147 32487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:34.120363 32487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:34.120385 32487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:34.121518 32487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:34.121556 32487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:34.121572 32487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:34.121583 32487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:34.121595 32487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:34.123080 32487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:38013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:34.123286 32487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:34.123459 32487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:34.128566 32493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:34.128578 32492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:34.128648 32487 server_base.cc:1034] running on GCE node
W20260929 19:05:34.128578 32495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:34.129007 32487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:34.129329 32487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:34.130482 32487 hybrid_clock.cc:648] HybridClock initialized: now 1790708734130444 us; error 53 us; skew 500 ppm
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.136085 32487 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:05:34.136582 32487 webserver.cc:460] Webserver started at http://127.26.78.254:39751/ using document root <none> and password file <none>
I20260929 19:05:34.136833 32487 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:34.136905 32487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:34.137055 32487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:34.138237 32487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "dc0d3116c1e941329b6a15e191908cf0"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "54d44c9d4c5df67aa71fdfe4cafeb304"
server_key_iv: "ba9f245285e76518c88e7d2dc8e8bc16"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.138614 32487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "dc0d3116c1e941329b6a15e191908cf0"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "54d44c9d4c5df67aa71fdfe4cafeb304"
server_key_iv: "ba9f245285e76518c88e7d2dc8e8bc16"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.139993 32487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:34.140798 32502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.141116 32487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:34.141227 32487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "dc0d3116c1e941329b6a15e191908cf0"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "54d44c9d4c5df67aa71fdfe4cafeb304"
server_key_iv: "ba9f245285e76518c88e7d2dc8e8bc16"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.141304 32487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:34.152415 32487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:34.154497 32487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:34.154670 32487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:34.158716 32487 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:38013
I20260929 19:05:34.158749 32554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:38013 every 8 connection(s)
I20260929 19:05:34.159121 32487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:34.159984 32555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:34.162141 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 32487
I20260929 19:05:34.162256 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:34.162405 32555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0: Bootstrap starting.
I20260929 19:05:34.162482 26939 external_mini_cluster.cc:1428] Setting key 7efe66b76677dc508d35f5cee0d4992e
I20260929 19:05:34.163107 32555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:34.163467 32555 log.cc:824] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:34.164178 32555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0: No bootstrap required, opened a new log
I20260929 19:05:34.166066 32555 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0d3116c1e941329b6a15e191908cf0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38013 } }
I20260929 19:05:34.166231 32555 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:34.166270 32555 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0d3116c1e941329b6a15e191908cf0, State: Initialized, Role: FOLLOWER
I20260929 19:05:34.166450 32555 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0d3116c1e941329b6a15e191908cf0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38013 } }
I20260929 19:05:34.166531 32555 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:34.166559 32555 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:34.166599 32555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:34.167284 32555 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0d3116c1e941329b6a15e191908cf0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38013 } }
I20260929 19:05:34.167418 32555 leader_election.cc:302] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc0d3116c1e941329b6a15e191908cf0; no voters:
I20260929 19:05:34.167630 32555 leader_election.cc:288] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:34.167660 32560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:34.167901 32555 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:34.167893 32560 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [term 1 LEADER]: Becoming Leader. State: Replica: dc0d3116c1e941329b6a15e191908cf0, State: Running, Role: LEADER
I20260929 19:05:34.168602 32560 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0d3116c1e941329b6a15e191908cf0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38013 } }
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:34.169236 32561 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc0d3116c1e941329b6a15e191908cf0. Latest consensus state: current_term: 1 leader_uuid: "dc0d3116c1e941329b6a15e191908cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0d3116c1e941329b6a15e191908cf0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38013 } } }
I20260929 19:05:34.169523 32561 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:34.169239 32562 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc0d3116c1e941329b6a15e191908cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0d3116c1e941329b6a15e191908cf0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38013 } } }
I20260929 19:05:34.169653 32562 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0 [sys.catalog]: This master's current role is: LEADER
W20260929 19:05:34.169811 32574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:05:34.169910 32574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:05:34.172371 32576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:34.172572 32576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:34.173944 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.163020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37262 (local address 127.26.78.254:38013)
0929 19:05:34.163204 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.163212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.163237 (+ 25us) server_negotiation.cc:407] Connection header received
0929 19:05:34.163680 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.163687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.163733 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.163889 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.164578 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.165466 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.166327 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.166957 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.167080 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.170035 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:34.170039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:34.170039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:34.170052 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:34.171918 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.172267 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.172270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.172270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.172383 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.172625 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.172625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.172626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.172949 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:34.173456 (+ 507us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:34.173532 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 19:05:34.173673 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260929 19:05:34.174336 32576 catalog_manager.cc:1347] Generated new cluster ID: 8a4e19af842d4131adf36d374d56966b
I20260929 19:05:34.174412 32576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:05:34.206254 32576 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:34.206812 32576 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:34.222524 32576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc0d3116c1e941329b6a15e191908cf0: Generated new TSK 0
I20260929 19:05:34.222754 32576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:05:34.258400 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38013
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:34.341396 32581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:34.341607 32581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:34.341630 32581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:34.342950 32581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:34.342993 32581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:34.343019 32581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:34.344594 32581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:34.344864 32581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:34.345103 32581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:34.350106 32587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:34.350219 32581 server_base.cc:1034] running on GCE node
W20260929 19:05:34.350103 32589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:34.350107 32586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:34.350553 32581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:34.350852 32581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:34.352006 32581 hybrid_clock.cc:648] HybridClock initialized: now 1790708734351988 us; error 33 us; skew 500 ppm
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.354640 32581 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:05:34.355147 32581 webserver.cc:460] Webserver started at http://127.26.78.193:44329/ using document root <none> and password file <none>
I20260929 19:05:34.355387 32581 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:34.355458 32581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:34.355607 32581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:34.356657 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "cee83d8ac4a04ac5882f63734d6f641f"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "af00e38f50367041f64c8a0ad5c07f94"
server_key_iv: "f6993c5a0030bdb001183ffff0c7c31b"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.357033 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "cee83d8ac4a04ac5882f63734d6f641f"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "af00e38f50367041f64c8a0ad5c07f94"
server_key_iv: "f6993c5a0030bdb001183ffff0c7c31b"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.358446 32581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:05:34.359222 32596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.359400 32581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:34.359500 32581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "cee83d8ac4a04ac5882f63734d6f641f"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "af00e38f50367041f64c8a0ad5c07f94"
server_key_iv: "f6993c5a0030bdb001183ffff0c7c31b"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.359581 32581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:34.380784 32581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:34.383366 32581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:34.383533 32581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:34.383782 32581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:34.384092 32581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:34.384148 32581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.384191 32581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:34.384212 32581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.389976 32581 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:34841
I20260929 19:05:34.390039 32709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:34841 every 8 connection(s)
I20260929 19:05:34.390384 32581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:05:34.393625 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 32581
I20260929 19:05:34.393739 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:34.393944 26939 external_mini_cluster.cc:1428] Setting key 852ac9a57a1c5a6bdc66a020ffea55be
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:34.401099 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.391172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:57263 (local address 127.26.78.254:38013)
0929 19:05:34.391242 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.391245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.391464 (+ 219us) server_negotiation.cc:407] Connection header received
0929 19:05:34.392455 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.392456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.392467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.392525 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.393184 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.393953 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.395148 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.395393 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.395490 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.397938 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:34.397940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:34.397940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:34.397949 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:34.399639 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.400074 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.400076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.400076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.400142 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.400394 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.400395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.400396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.400543 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:34.400619 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:34.400908 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 19:05:34.400998 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:34.401209 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.391145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38013 (local address 127.26.78.193:57263)
0929 19:05:34.391323 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:34.391370 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:34.392345 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:34.392602 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:34.392605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:34.392896 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:34.393089 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:34.393094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.394060 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.394061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:34.395043 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:34.395046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.395500 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.395501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:34.395635 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.396191 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:34.396204 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:34.397776 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:34.399742 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:34.399749 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:34.399751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:34.399987 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:34.400216 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:34.400217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:34.400217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:34.400319 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:34.400616 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:34.400672 (+ 56us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:34.400789 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 19:05:34.400876 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 19:05:34.400996 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260929 19:05:34.401806 32710 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38013
I20260929 19:05:34.401975 32710 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:34.402179 32710 heartbeater.cc:511] Master 127.26.78.254:38013 requested a full tablet report, sending...
I20260929 19:05:34.402792 32519 ts_manager.cc:194] Registered new tserver with Master: cee83d8ac4a04ac5882f63734d6f641f (127.26.78.193:34841)
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
I20260929 19:05:34.403602 32519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:57263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:05:34.420747 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38013
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:34.501956 32715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:34.502220 32715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:34.502257 32715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:34.503381 32715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:34.503418 32715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:34.503444 32715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:34.504849 32715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:34.505116 32715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:34.505321 32715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:34.510468 32723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:34.510522 32721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:34.510462 32720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:34.510699 32715 server_base.cc:1034] running on GCE node
I20260929 19:05:34.510968 32715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:34.511230 32715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:34.512424 32715 hybrid_clock.cc:648] HybridClock initialized: now 1790708734512419 us; error 37 us; skew 500 ppm
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.515003 32715 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:05:34.515437 32715 webserver.cc:460] Webserver started at http://127.26.78.194:42829/ using document root <none> and password file <none>
I20260929 19:05:34.515679 32715 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:34.515748 32715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:34.515906 32715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:34.516940 32715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "1233d807b5cd4b83b017c6dcc2c89796"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "a428f4fba7eab2d0daf5db3d0255b8f1"
server_key_iv: "6d451506474f43ecab3b871da9111d24"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.517300 32715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "1233d807b5cd4b83b017c6dcc2c89796"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "a428f4fba7eab2d0daf5db3d0255b8f1"
server_key_iv: "6d451506474f43ecab3b871da9111d24"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.518697 32715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:05:34.519440 32730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.519618 32715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:34.519716 32715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "1233d807b5cd4b83b017c6dcc2c89796"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "a428f4fba7eab2d0daf5db3d0255b8f1"
server_key_iv: "6d451506474f43ecab3b871da9111d24"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.519794 32715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:34.532958 32715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:34.534911 32715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:34.535086 32715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:34.535336 32715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:34.535661 32715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:34.535719 32715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.535763 32715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:34.535791 32715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.541694 32715 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:34451
I20260929 19:05:34.541744 375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:34451 every 8 connection(s)
I20260929 19:05:34.542124 32715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:34.545640 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 32715
I20260929 19:05:34.545732 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:34.545986 26939 external_mini_cluster.cc:1428] Setting key 8e02ded18dc098faf0dff117287f92db
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:34.553437 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.542850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:46323 (local address 127.26.78.254:38013)
0929 19:05:34.542933 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.542935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.543133 (+ 198us) server_negotiation.cc:407] Connection header received
0929 19:05:34.544154 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.544155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.544165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.544268 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.544862 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.545419 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.547932 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.548121 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.548188 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.550584 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:34.550585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:34.550586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:34.550593 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:34.552035 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.552455 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.552456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.552457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.552515 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.552777 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.552778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.552778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.552891 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:34.552965 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:34.553249 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 19:05:34.553348 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 19:05:34.553618 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.542813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38013 (local address 127.26.78.194:46323)
0929 19:05:34.542990 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:34.543043 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:34.544060 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:34.544346 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:34.544350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:34.544602 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:34.544790 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:34.544793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.545498 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.545499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:34.546593 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:34.546596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.548213 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.548215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:34.548380 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.548898 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:34.548909 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:34.550482 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:34.552125 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:34.552132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:34.552135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:34.552360 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:34.552585 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:34.552586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:34.552586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:34.552700 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:34.552967 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:34.552974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:34.553111 (+ 137us) client_negotiation.cc:769] Sending connection context
0929 19:05:34.553214 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 19:05:34.553372 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
I20260929 19:05:34.554172 376 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38013
I20260929 19:05:34.554265 376 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:34.554417 376 heartbeater.cc:511] Master 127.26.78.254:38013 requested a full tablet report, sending...
I20260929 19:05:34.554777 32519 ts_manager.cc:194] Registered new tserver with Master: 1233d807b5cd4b83b017c6dcc2c89796 (127.26.78.194:34451)
I20260929 19:05:34.555111 32519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:46323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:05:34.572216 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38013
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:34.654595 381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:34.654803 381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:34.654827 381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:34.656106 381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:34.656147 381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:34.656173 381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:34.657734 381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:34.658020 381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:34.658254 381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:34.663543 387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:34.663617 386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:34.663550 389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:34.663825 381 server_base.cc:1034] running on GCE node
I20260929 19:05:34.664072 381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:34.664318 381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:34.665488 381 hybrid_clock.cc:648] HybridClock initialized: now 1790708734665486 us; error 40 us; skew 500 ppm
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.668040 381 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:05:34.668473 381 webserver.cc:460] Webserver started at http://127.26.78.195:35877/ using document root <none> and password file <none>
I20260929 19:05:34.668716 381 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:34.668761 381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:34.668865 381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:34.669974 381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "7fde2f24179f44e1b6e474a518c09fdf"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "0038344571b0c9750cf49b6e79161950"
server_key_iv: "772db67eedc406e2508269f7fb5c515f"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.670280 381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "7fde2f24179f44e1b6e474a518c09fdf"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "0038344571b0c9750cf49b6e79161950"
server_key_iv: "772db67eedc406e2508269f7fb5c515f"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.671512 381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:34.672201 396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.672379 381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.672477 381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "7fde2f24179f44e1b6e474a518c09fdf"
format_stamp: "Formatted at 2026-09-29 19:05:34 on dist-test-slave-xz51"
server_key: "0038344571b0c9750cf49b6e79161950"
server_key_iv: "772db67eedc406e2508269f7fb5c515f"
server_key_version: "encryptionkey@0"
I20260929 19:05:34.672557 381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:34.699448 381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:34.701447 381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:34.701627 381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:34.701951 381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:34.702286 381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:34.702350 381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.702394 381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:34.702421 381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:34.708380 381 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:38373
I20260929 19:05:34.708480 509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:38373 every 8 connection(s)
I20260929 19:05:34.708797 381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:05:34.717549 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 381
I20260929 19:05:34.717656 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:34.717900 26939 external_mini_cluster.cc:1428] Setting key 2a121e6f5b9ae35f26deb144533c337a
I20260929 19:05:34.718116 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.709536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:56511 (local address 127.26.78.254:38013)
0929 19:05:34.709607 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.709609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.709852 (+ 243us) server_negotiation.cc:407] Connection header received
0929 19:05:34.710814 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.710816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.710826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.710882 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.711568 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.712077 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.713032 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.713211 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.713262 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.715371 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:34.715372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:34.715372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:34.715379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:34.716591 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.716995 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.716996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.716997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.717051 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.717267 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.717267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.717268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.717395 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:34.717445 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:34.717913 (+ 468us) server_negotiation.cc:299] Negotiation successful
0929 19:05:34.718012 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:05:34.718307 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.709531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38013 (local address 127.26.78.195:56511)
0929 19:05:34.709690 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:34.709726 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:34.710704 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:34.710961 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:34.710964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:34.711221 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:34.711477 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:34.711482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.712160 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.712161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:34.712956 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:34.712958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.713285 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.713286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:34.713394 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.713923 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:34.713933 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:34.715284 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:34.716666 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:34.716671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:34.716673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:34.716919 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:34.717114 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:34.717115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:34.717115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:34.717211 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:34.717487 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:34.717495 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:34.717678 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 19:05:34.717854 (+ 176us) client_negotiation.cc:240] Negotiation successful
0929 19:05:34.718044 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260929 19:05:34.718808 510 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38013
I20260929 19:05:34.718904 510 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:34.719106 510 heartbeater.cc:511] Master 127.26.78.254:38013 requested a full tablet report, sending...
I20260929 19:05:34.719502 32518 ts_manager.cc:194] Registered new tserver with Master: 7fde2f24179f44e1b6e474a518c09fdf (127.26.78.195:38373)
I20260929 19:05:34.719849 32518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:56511
I20260929 19:05:34.729692 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/test-admin.keytab.
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.747354 26939 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 19:05:34.748991 517 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:34.748232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.26.78.193:34841)
0929 19:05:34.748400 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.748404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.748415 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:05:34.748439 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.748441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.748455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.748526 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.748882 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46506: BlockingRecv error: recv got EOF from 127.0.0.1:46506 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
W20260929 19:05:34.749075 26939 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5cc' of type FILE
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.750134 26939 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.751026 26939 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:34.758708 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.751558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46512 (local address 127.26.78.193:34841)
0929 19:05:34.751641 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.751643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.751700 (+ 57us) server_negotiation.cc:407] Connection header received
0929 19:05:34.751743 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.751744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.751754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.751808 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.752185 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.752917 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.753597 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.753999 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.754057 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.756118 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:34.756126 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:34.756127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:34.756133 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:34.757549 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.757938 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.757941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.757942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.757997 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.758174 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.758174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.758175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.758309 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:34.758412 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:34.758486 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 19:05:34.758547 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 19:05:34.760531 517 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:34.760027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.26.78.193:34841)
0929 19:05:34.760081 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.760083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.760112 (+ 29us) server_negotiation.cc:407] Connection header received
0929 19:05:34.760173 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.760173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.760182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.760219 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.760491 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46526: BlockingRecv error: recv got EOF from 127.0.0.1:46526 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260929 19:05:34.760604 26939 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5cc' of type FILE
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.761557 26939 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.762499 26939 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:34.769603 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.763013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46532 (local address 127.26.78.193:34841)
0929 19:05:34.763079 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.763081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.763143 (+ 62us) server_negotiation.cc:407] Connection header received
0929 19:05:34.763217 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.763218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.763228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.763277 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.763657 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.764146 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.764859 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.765064 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.765142 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.767153 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:34.767154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:34.767155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:34.767162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:34.768467 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.768821 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.768822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.768823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.768880 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.769112 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.769113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.769113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.769212 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:34.769324 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:34.769421 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:05:34.769501 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260929 19:05:34.771127 517 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:05:34.770687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.26.78.193:34841)
0929 19:05:34.770742 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.770744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.770752 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:34.770795 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.770796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.770803 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.770849 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.771059 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46538: BlockingRecv error: recv got EOF from 127.0.0.1:46538 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260929 19:05:34.771222 26939 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790708710206825-26939-0/krb5kdc/krb5cc' of type FILE
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.772149 26939 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:34.773000 26939 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[32474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:05:34.779997 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:34.773473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.26.78.193:34841)
0929 19:05:34.773504 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:34.773506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:34.773535 (+ 29us) server_negotiation.cc:407] Connection header received
0929 19:05:34.773587 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:34.773588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:34.773594 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:34.773641 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:34.774060 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.774514 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.775347 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:34.775530 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:34.775585 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:34.777574 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:34.777575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:34.777576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:34.777581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:34.778920 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.779251 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.779252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.779252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.779305 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:34.779549 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:34.779550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:34.779550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:34.779638 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:34.779709 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:34.779822 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 19:05:34.779874 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 19:05:34.780592 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 32581
I20260929 19:05:34.785620 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 32715
I20260929 19:05:34.789716 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 381
I20260929 19:05:34.794055 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 32487
2026-09-29T19:05:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1912 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[530](info): set up 2 sockets
Sep 29 19:05:34 dist-test-slave-xz51 krb5kdc[530](info): commencing operation
krb5kdc: starting...
W20260929 19:05:35.834306 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:05:35 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:05:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab.
Entry for principal oryx/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab.
I20260929 19:05:35.953548 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44725
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab
--principal=oryx/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:36.035403 543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:36.035610 543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:36.035632 543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:36.036810 543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:36.036855 543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:36.036870 543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:36.036882 543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:36.036937 543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:36.038401 543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab
--principal=oryx/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:36.038664 543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:36.038906 543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:36.043732 551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.043916 549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.044025 543 server_base.cc:1034] running on GCE node
W20260929 19:05:36.043758 548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.044427 543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:36.044731 543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:36.045883 543 hybrid_clock.cc:648] HybridClock initialized: now 1790708736045867 us; error 33 us; skew 500 ppm
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:36.048566 543 init.cc:377] Logged in from keytab as oryx/127.26.78.254@KRBTEST.COM (short username oryx)
I20260929 19:05:36.049062 543 webserver.cc:460] Webserver started at http://127.26.78.254:45057/ using document root <none> and password file <none>
I20260929 19:05:36.049340 543 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.049413 543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.049572 543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:36.050727 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "6abeb109cf0d4a4da444f9e29f8fd826"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "c07e6b02c7c32829be4d2901f4ca1bcc"
server_key_iv: "2a9c32ef9659b03e3ecf6c4a24cf0699"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.051100 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "6abeb109cf0d4a4da444f9e29f8fd826"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "c07e6b02c7c32829be4d2901f4ca1bcc"
server_key_iv: "2a9c32ef9659b03e3ecf6c4a24cf0699"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.052470 543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:05:36.053216 558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.053398 543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:36.053491 543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "6abeb109cf0d4a4da444f9e29f8fd826"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "c07e6b02c7c32829be4d2901f4ca1bcc"
server_key_iv: "2a9c32ef9659b03e3ecf6c4a24cf0699"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.053579 543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:36.066529 543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:36.068986 543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:36.069162 543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:36.073248 543 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:44725
I20260929 19:05:36.073287 610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:44725 every 8 connection(s)
I20260929 19:05:36.073653 543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:05:36.074702 611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:36.077018 611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: Bootstrap starting.
I20260929 19:05:36.077620 611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:36.077945 611 log.cc:824] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:36.078584 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 543
I20260929 19:05:36.078689 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:05:36.078833 611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: No bootstrap required, opened a new log
I20260929 19:05:36.078897 26939 external_mini_cluster.cc:1428] Setting key ea544128ede902039467032bdee031e6
I20260929 19:05:36.081537 611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } }
I20260929 19:05:36.081688 611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:36.081712 611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abeb109cf0d4a4da444f9e29f8fd826, State: Initialized, Role: FOLLOWER
I20260929 19:05:36.081890 611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } }
I20260929 19:05:36.081979 611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:05:36.082005 611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:05:36.082077 611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:36.082687 611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } }
I20260929 19:05:36.082788 611 leader_election.cc:302] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6abeb109cf0d4a4da444f9e29f8fd826; no voters:
I20260929 19:05:36.083041 611 leader_election.cc:288] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:05:36.083065 616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:36.083177 616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 1 LEADER]: Becoming Leader. State: Replica: 6abeb109cf0d4a4da444f9e29f8fd826, State: Running, Role: LEADER
I20260929 19:05:36.083271 616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } }
I20260929 19:05:36.083326 611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:36.083590 616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } }
I20260929 19:05:36.083659 616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:36.083791 616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6abeb109cf0d4a4da444f9e29f8fd826. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } }
I20260929 19:05:36.083863 616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:36.084698 621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:05:36.085323 621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:05:36.087280 621 catalog_manager.cc:1347] Generated new cluster ID: 0cbae24eb7c14371b06008a758666cf6
I20260929 19:05:36.087347 621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:05:36.091570 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.079454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.26.78.254:44725)
0929 19:05:36.079651 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.079665 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.079690 (+ 25us) server_negotiation.cc:407] Connection header received
0929 19:05:36.080212 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.080220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.080271 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.080466 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.081455 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.082866 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.084016 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.084744 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.084853 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.087899 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.087903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.087903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.087916 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.089914 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.090331 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.090334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.090334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.090433 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.090653 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.090654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.090654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.090940 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.091096 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.091224 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.091375 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":120,"threads_started":1}
I20260929 19:05:36.097862 621 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:05:36.098587 621 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:05:36.101643 621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: Generated new TSK 0
I20260929 19:05:36.101862 621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab.
Entry for principal oryx/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab.
I20260929 19:05:36.134256 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44725
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab
--principal=oryx/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:36.218648 637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:36.218900 637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:36.218942 637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:36.220131 637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:36.220211 637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:36.220258 637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:36.221771 637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab
--principal=oryx/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:36.222100 637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:36.222348 637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:36.227510 642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.227617 643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.227645 645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.228039 637 server_base.cc:1034] running on GCE node
I20260929 19:05:36.228288 637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:36.228561 637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:36.229727 637 hybrid_clock.cc:648] HybridClock initialized: now 1790708736229711 us; error 27 us; skew 500 ppm
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:36.232316 637 init.cc:377] Logged in from keytab as oryx/127.26.78.193@KRBTEST.COM (short username oryx)
I20260929 19:05:36.232749 637 webserver.cc:460] Webserver started at http://127.26.78.193:34745/ using document root <none> and password file <none>
I20260929 19:05:36.232991 637 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.233062 637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.233196 637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:36.234416 637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "1663f4cc14bf4d49b8f09597b73caf0b"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "f8ce6d371f8411bb93c61f5544d142b1"
server_key_iv: "10f86c0b5829575288dbd0e911c370fd"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.234784 637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "1663f4cc14bf4d49b8f09597b73caf0b"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "f8ce6d371f8411bb93c61f5544d142b1"
server_key_iv: "10f86c0b5829575288dbd0e911c370fd"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.236114 637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:36.236888 652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.237068 637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:36.237174 637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "1663f4cc14bf4d49b8f09597b73caf0b"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "f8ce6d371f8411bb93c61f5544d142b1"
server_key_iv: "10f86c0b5829575288dbd0e911c370fd"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.237258 637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:36.257210 637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:36.259217 637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:36.259392 637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:36.259636 637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:36.259972 637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:36.260030 637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.260070 637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:36.260092 637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.265915 637 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:40115
I20260929 19:05:36.265936 765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:40115 every 8 connection(s)
I20260929 19:05:36.266290 637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:05:36.269572 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 637
I20260929 19:05:36.269706 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:05:36.269989 26939 external_mini_cluster.cc:1428] Setting key d2e4471d35ae3b91b9ec357f6efb689b
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:05:36.276628 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.267040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:32807 (local address 127.26.78.254:44725)
0929 19:05:36.267086 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.267088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.267282 (+ 194us) server_negotiation.cc:407] Connection header received
0929 19:05:36.267863 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.267864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.267876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.267929 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.268914 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.269462 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.270903 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.271113 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.271202 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.273631 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.273633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.273633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.273659 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.275120 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.275616 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.275616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.275617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.275666 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.275975 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.275975 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.275975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.276075 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.276124 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.276491 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.276563 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:05:36.276971 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.266995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:32807)
0929 19:05:36.267150 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.267194 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.267708 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.268011 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.268015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.268594 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.268806 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.268811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.269565 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.269566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.270799 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.270802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.271213 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.271214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.271350 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.271892 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.271907 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.273507 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.275208 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.275216 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.275219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.275536 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.275737 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.275739 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.275739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.275905 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.276143 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.276150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.276343 (+ 193us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.276478 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.276669 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260929 19:05:36.277550 766 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44725
I20260929 19:05:36.277649 766 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:36.277933 766 heartbeater.cc:511] Master 127.26.78.254:44725 requested a full tablet report, sending...
I20260929 19:05:36.278506 575 ts_manager.cc:194] Registered new tserver with Master: 1663f4cc14bf4d49b8f09597b73caf0b (127.26.78.193:40115)
I20260929 19:05:36.279116 575 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:32807
WARNING: no policy specified for oryx/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab.
Entry for principal oryx/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab.
I20260929 19:05:36.298796 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44725
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab
--principal=oryx/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:36.379663 771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:36.379922 771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:36.379958 771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:36.381171 771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:36.381264 771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:36.381306 771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:36.382897 771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab
--principal=oryx/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:36.383401 771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:36.383705 771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:36.388888 779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.388993 771 server_base.cc:1034] running on GCE node
W20260929 19:05:36.388883 777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.388888 776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.389374 771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:36.389619 771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:36.390789 771 hybrid_clock.cc:648] HybridClock initialized: now 1790708736390759 us; error 60 us; skew 500 ppm
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:36.393224 771 init.cc:377] Logged in from keytab as oryx/127.26.78.194@KRBTEST.COM (short username oryx)
I20260929 19:05:36.393647 771 webserver.cc:460] Webserver started at http://127.26.78.194:41421/ using document root <none> and password file <none>
I20260929 19:05:36.393952 771 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.394022 771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.394155 771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:36.395196 771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "d86b1258564348998c1cc67688a4111d"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "74389f89eba68d7c93124d65bd89613b"
server_key_iv: "c841924cf48ff9ebca94c82ca70e1ad2"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.395555 771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "d86b1258564348998c1cc67688a4111d"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "74389f89eba68d7c93124d65bd89613b"
server_key_iv: "c841924cf48ff9ebca94c82ca70e1ad2"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.396853 771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:36.397531 786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.397707 771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:36.397866 771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d86b1258564348998c1cc67688a4111d"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "74389f89eba68d7c93124d65bd89613b"
server_key_iv: "c841924cf48ff9ebca94c82ca70e1ad2"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.397956 771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:36.409565 771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:36.411556 771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:36.411727 771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:36.411968 771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:36.412279 771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:36.412339 771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.412381 771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:36.412427 771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.418308 771 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:33085
I20260929 19:05:36.418399 900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:33085 every 8 connection(s)
I20260929 19:05:36.418743 771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:05:36.423400 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 771
I20260929 19:05:36.423485 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:05:36.423736 26939 external_mini_cluster.cc:1428] Setting key 5e12b5a3c18ca756b938674f97a34b11
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:05:36.428864 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.419479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:38439 (local address 127.26.78.254:44725)
0929 19:05:36.419531 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.419533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.419743 (+ 210us) server_negotiation.cc:407] Connection header received
0929 19:05:36.420230 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.420231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.420240 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.420311 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.421226 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.421737 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.422782 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.422957 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.423008 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.425724 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.425726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.425726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.425732 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.427471 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.427863 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.427865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.427865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.427931 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.428161 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.428162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.428162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.428315 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.428393 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.428661 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.428743 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:05:36.428932 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.419443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:38439)
0929 19:05:36.419599 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.419640 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.420140 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.420376 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.420379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.420942 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.421136 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.421141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.421875 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.421877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.422697 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.422700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.423031 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.423032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.423132 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.423750 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.423759 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.425632 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.427553 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.427559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.427561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.427784 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.428000 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.428000 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.428000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.428095 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.428389 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.428399 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.428517 (+ 118us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.428606 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.428730 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260929 19:05:36.429584 901 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44725
I20260929 19:05:36.429701 901 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:36.429930 901 heartbeater.cc:511] Master 127.26.78.254:44725 requested a full tablet report, sending...
I20260929 19:05:36.430353 575 ts_manager.cc:194] Registered new tserver with Master: d86b1258564348998c1cc67688a4111d (127.26.78.194:33085)
I20260929 19:05:36.430879 575 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:38439
WARNING: no policy specified for oryx/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab.
Entry for principal oryx/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab.
I20260929 19:05:36.452935 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44725
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab
--principal=oryx/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:36.533571 906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:36.533782 906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:36.533804 906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:36.534976 906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:36.535018 906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:36.535044 906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:36.536388 906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab
--principal=oryx/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:36.536569 906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:36.536736 906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:36.541985 911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.541992 912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.541992 914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.542212 906 server_base.cc:1034] running on GCE node
I20260929 19:05:36.542390 906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:36.542621 906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:36.543761 906 hybrid_clock.cc:648] HybridClock initialized: now 1790708736543746 us; error 26 us; skew 500 ppm
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:36.546190 906 init.cc:377] Logged in from keytab as oryx/127.26.78.195@KRBTEST.COM (short username oryx)
I20260929 19:05:36.546562 906 webserver.cc:460] Webserver started at http://127.26.78.195:40693/ using document root <none> and password file <none>
I20260929 19:05:36.546814 906 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.546887 906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.547035 906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:36.548115 906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "a526a417a8e146319a0e1b6ea63c9004"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "7713b0d1c39bf9cc045e99120f8dc380"
server_key_iv: "507ece0063ade1a4e43483c8a4e7ed59"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.548485 906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "a526a417a8e146319a0e1b6ea63c9004"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "7713b0d1c39bf9cc045e99120f8dc380"
server_key_iv: "507ece0063ade1a4e43483c8a4e7ed59"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.549923 906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260929 19:05:36.550740 921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.550923 906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:36.551024 906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "a526a417a8e146319a0e1b6ea63c9004"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "7713b0d1c39bf9cc045e99120f8dc380"
server_key_iv: "507ece0063ade1a4e43483c8a4e7ed59"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.551110 906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:36.560359 906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:36.562525 906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:36.562688 906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:36.562925 906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:36.563232 906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:36.563288 906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.563321 906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:36.563349 906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.569161 906 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:41141
I20260929 19:05:36.569252 1034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:41141 every 8 connection(s)
I20260929 19:05:36.569569 906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:05:36.578109 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 906
I20260929 19:05:36.578228 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:05:36.578426 26939 external_mini_cluster.cc:1428] Setting key 5d399afbe9b1d3e62e74b33825a7e9aa
I20260929 19:05:36.579124 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.570336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:40147 (local address 127.26.78.254:44725)
0929 19:05:36.570398 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.570400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.570616 (+ 216us) server_negotiation.cc:407] Connection header received
0929 19:05:36.571184 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.571185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.571197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.571255 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.572223 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.572719 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.573752 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.574014 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.574101 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.576239 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.576240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.576240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.576249 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.577656 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.578101 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.578102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.578103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.578167 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.578431 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.578432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.578433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.578589 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.578651 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.578953 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.579035 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:05:36.579316 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.570335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:40147)
0929 19:05:36.570495 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.570532 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.571059 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.571339 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.571344 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.571923 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.572109 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.572114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.572825 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.572826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.573647 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.573650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.574110 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.574111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.574242 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.574786 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.574794 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.576153 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.577731 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.577736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.577738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.578017 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.578240 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.578241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.578241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.578352 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.578642 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.578649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.578787 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.578915 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.579072 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260929 19:05:36.579696 1035 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44725
I20260929 19:05:36.579805 1035 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:36.579981 1035 heartbeater.cc:511] Master 127.26.78.254:44725 requested a full tablet report, sending...
I20260929 19:05:36.580369 575 ts_manager.cc:194] Registered new tserver with Master: a526a417a8e146319a0e1b6ea63c9004 (127.26.78.195:41141)
I20260929 19:05:36.580734 575 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:40147
I20260929 19:05:36.590458 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.78.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.253.keytab.
Entry for principal oryx/127.26.78.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.253.keytab.
I20260929 19:05:36.617291 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:36035
--webserver_interface=127.26.78.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.253.keytab
--principal=oryx/127.26.78.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44725,127.26.78.253:36035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:36.700014 1040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:36.700222 1040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:36.700244 1040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:36.701453 1040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:36.701498 1040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:36.701511 1040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:36.701524 1040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:36.701581 1040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:36.703054 1040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44725,127.26.78.253:36035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.253.keytab
--principal=oryx/127.26.78.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:36035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.78.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
W20260929 19:05:36.703188 1040 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.78.254:44725,127.26.78.253:36035'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 19:05:36.703336 1040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:36.703564 1040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:36.708467 1046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.708532 1040 server_base.cc:1034] running on GCE node
W20260929 19:05:36.708494 1045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.708491 1048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.708911 1040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:36.709216 1040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:36.710358 1040 hybrid_clock.cc:648] HybridClock initialized: now 1790708736710349 us; error 31 us; skew 500 ppm
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:36.712993 1040 init.cc:377] Logged in from keytab as oryx/127.26.78.253@KRBTEST.COM (short username oryx)
I20260929 19:05:36.713451 1040 webserver.cc:460] Webserver started at http://127.26.78.253:37043/ using document root <none> and password file <none>
I20260929 19:05:36.713689 1040 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.713761 1040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.713928 1040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:36.715040 1040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/instance:
uuid: "7842c1473cb44de6b63e14ddc22c71e6"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "e160d0074c82b511ce0704989c7b1332"
server_key_iv: "c22eb60119f218b564fff3e672dbc9e4"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.715410 1040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal/instance:
uuid: "7842c1473cb44de6b63e14ddc22c71e6"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "e160d0074c82b511ce0704989c7b1332"
server_key_iv: "c22eb60119f218b564fff3e672dbc9e4"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.716852 1040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:05:36.717664 1055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.717867 1040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:36.717957 1040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
uuid: "7842c1473cb44de6b63e14ddc22c71e6"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "e160d0074c82b511ce0704989c7b1332"
server_key_iv: "c22eb60119f218b564fff3e672dbc9e4"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.718009 1040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:36.731505 1040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:36.733799 1040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:36.733982 1040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:36.737624 1040 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.253:36035
I20260929 19:05:36.737692 1107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.253:36035 every 8 connection(s)
I20260929 19:05:36.738039 1040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
I20260929 19:05:36.739064 1108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:36.740702 1108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:05:36.742285 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 1040
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.253@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.78.253@KRBTEST.COM
I20260929 19:05:36.750646 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.741521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.26.78.254:44725)
0929 19:05:36.741580 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.741582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.741862 (+ 280us) server_negotiation.cc:407] Connection header received
0929 19:05:36.742355 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.742356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.742369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.742431 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.743233 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.743774 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.744740 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.744932 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.744986 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.747454 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.747455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.747456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.747463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.749093 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.749468 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.749469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.749470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.749540 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.749898 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.749900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.749900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.750115 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.750196 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.750509 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.750578 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:05:36.750977 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.741464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37528)
0929 19:05:36.741697 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.741739 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.742242 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.742506 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.742510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.742836 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.743140 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.743164 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.743880 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.743881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.744606 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.744608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.745015 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.745016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.745168 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.745763 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.745771 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.747352 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.749185 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.749190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.749192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.749390 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.749644 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.749645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.749645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.749767 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.750199 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.750205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.750333 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.750477 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.750736 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260929 19:05:36.751745 1108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:05:36.751981 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.742806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.26.78.253:36035)
0929 19:05:36.742965 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.742969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.743001 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:05:36.743076 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.743079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.743097 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.743165 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.743773 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.744589 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.745555 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.745984 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.746040 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.748438 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.748445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.748446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.748452 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.750315 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.750707 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.750711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.750711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.750831 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.751059 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.751060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.751060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.751235 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.751410 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.751718 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.751803 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.253@KRBTEST.COM for oryx/127.26.78.253@KRBTEST.COM
I20260929 19:05:36.758184 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.752011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.0.0.1:38354)
0929 19:05:36.752074 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.752084 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.752122 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.752377 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.752378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.752522 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.752582 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.752584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.753226 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.753227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.753727 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.753728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.754154 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.754155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.754225 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.754587 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.754591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.755731 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.757167 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.757168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.757169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.757334 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.757546 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.757547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.757547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.757599 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.757878 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.757879 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.757918 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.757983 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.758078 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 19:05:36.758239 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.752069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.26.78.253:36035)
0929 19:05:36.752262 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.752264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.752272 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:36.752292 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.752292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.752303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.752357 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.752678 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.753144 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.753858 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.754081 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.754159 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.755833 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.755834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.755835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.755841 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.757064 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.757419 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.757420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.757421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.757473 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.757667 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.757668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.757668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.757777 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.757885 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.758013 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.758127 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":177,"thread_start_us":110,"threads_started":1}
I20260929 19:05:36.759640 1108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Bootstrap starting.
I20260929 19:05:36.760411 1108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:36.760643 1108 log.cc:824] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:36.761327 1108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: No bootstrap required, opened a new log
I20260929 19:05:36.762751 1108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } }
I20260929 19:05:36.762878 1108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:36.762902 1108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7842c1473cb44de6b63e14ddc22c71e6, State: Initialized, Role: FOLLOWER
I20260929 19:05:36.763026 1108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } }
I20260929 19:05:36.763271 1115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } } }
I20260929 19:05:36.763345 1108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:36.763352 1115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:05:36.765046 1126 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:05:36.765108 1126 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:05:36.766817 1040 master_runner.cc:428] Detected that this master 7842c1473cb44de6b63e14ddc22c71e6 is joining an existing cluster
I20260929 19:05:36.766866 1040 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.78.253:36035
I20260929 19:05:36.774461 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.768067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.0.0.1:38366)
0929 19:05:36.768193 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.768201 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.768252 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.768393 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.768393 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.768677 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.768754 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.768756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.769390 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.769392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.770057 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.770059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.770494 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.770495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.770550 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.771030 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.771036 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.771370 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.773283 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.773284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.773284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.773451 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.773666 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.773667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.773667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.773724 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.774015 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.774015 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.774056 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.774172 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.774336 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260929 19:05:36.774474 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.768101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38366 (local address 127.26.78.253:36035)
0929 19:05:36.768143 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.768146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.768302 (+ 156us) server_negotiation.cc:407] Connection header received
0929 19:05:36.768318 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.768318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.768326 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.768364 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.768837 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.769308 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.770157 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.770408 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.770484 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.771673 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.771674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.771674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.771680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.773201 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.773546 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.773547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.773547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.773600 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.773794 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.773795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.773796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.773945 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.774023 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.774197 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.774395 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":22}
I20260929 19:05:36.775554 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.767954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.26.78.254:44725)
0929 19:05:36.768017 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.768020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.768152 (+ 132us) server_negotiation.cc:407] Connection header received
0929 19:05:36.768258 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.768259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.768271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.768326 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.768869 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.769638 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.770779 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.771082 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.771125 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.772199 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.772200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.772201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.772208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.773981 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.774424 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.774426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.774426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.774482 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.775085 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.775086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.775086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.775198 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.775253 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.775437 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.775495 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:05:36.775854 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.767854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37534)
0929 19:05:36.768085 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.768096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.768183 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.768344 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.768345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.768674 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.768768 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.768770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.769739 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.769740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.770674 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.770677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.771203 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.771204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.771267 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.771677 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.771682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.772094 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.774072 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.774073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.774073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.774336 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.774965 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.774965 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.774966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.775030 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.775271 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.775272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.775340 (+ 68us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.775422 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.775519 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":85,"threads_started":1}
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.254@KRBTEST.COM for oryx/127.26.78.253@KRBTEST.COM
I20260929 19:05:36.785930 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.778162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.0.0.1:38382)
0929 19:05:36.778334 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.778379 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.778452 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.778596 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.778599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.778943 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.779012 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.779014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.779635 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.779636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.780580 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.780582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.780947 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.780948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.781026 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.781442 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.781447 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.782928 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.784769 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.784776 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.784778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.784997 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.785231 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.785231 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.785232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.785287 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.785568 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.785576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.785633 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.785706 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.785800 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":58,"threads_started":1}
I20260929 19:05:36.785949 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.778253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.26.78.253:36035)
0929 19:05:36.778291 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.778293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.778466 (+ 173us) server_negotiation.cc:407] Connection header received
0929 19:05:36.778526 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.778527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.778536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.778582 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.779099 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.779552 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.780689 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.780867 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.780923 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.783025 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.783026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.783027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.783034 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.784686 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.785099 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.785100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.785101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.785162 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.785352 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.785353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.785353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.785492 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.785563 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.785754 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.785858 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:05:36.786595 575 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.78.253:36035: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6"
member_type: NON_VOTER
last_known_addr {
host: "127.26.78.253"
port: 36035
}
attrs {
promote: true
}
}
dest_uuid: "6abeb109cf0d4a4da444f9e29f8fd826"
cas_config_opid_index: -1
I20260929 19:05:36.786943 575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } }
I20260929 19:05:36.787493 1135 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7842c1473cb44de6b63e14ddc22c71e6 (127.26.78.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } }
I20260929 19:05:36.787774 1082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:36.788408 1135 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.78.253:36035
I20260929 19:05:36.788560 1135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } }
I20260929 19:05:36.788704 1135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:36.788714 1040 master.cc:537] Master@127.26.78.253:36035 shutting down...
I20260929 19:05:36.788970 1082 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Refusing update from remote peer 6abeb109cf0d4a4da444f9e29f8fd826: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 19:05:36.789384 1135 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 19:05:36.790125 562 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 19:05:36.790729 562 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 -> Peer 7842c1473cb44de6b63e14ddc22c71e6 (127.26.78.253:36035): Couldn't send request to peer 7842c1473cb44de6b63e14ddc22c71e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 19:05:36.792122 1040 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:36.792222 1040 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:36.792277 1040 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: stopping tablet replica
I20260929 19:05:36.804817 1040 master.cc:552] Master@127.26.78.253:36035 shutdown complete.
I20260929 19:05:36.804883 1040 master_runner.cc:305] Clearing existing system tablet
I20260929 19:05:36.805032 1040 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.805078 1040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.805745 1040 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:36.806340 1140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.806491 1040 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:36.806550 1040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.806573 1040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
uuid: "7842c1473cb44de6b63e14ddc22c71e6"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "e160d0074c82b511ce0704989c7b1332"
server_key_iv: "c22eb60119f218b564fff3e672dbc9e4"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.807219 1040 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:36.808861 1040 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 19:05:36.808929 1040 log.cc:1196] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 19:05:36.809192 1040 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Deleting consensus metadata
I20260929 19:05:36.809346 1040 master_runner.cc:315] Copying system tablet from 127.26.78.254:44725
I20260929 19:05:36.810160 1040 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: tablet copy: Beginning tablet copy session from remote peer at address 127.26.78.254:44725
I20260929 19:05:36.816955 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.810623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.26.78.254:44725)
0929 19:05:36.810673 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.810675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.810739 (+ 64us) server_negotiation.cc:407] Connection header received
0929 19:05:36.810821 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.810822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.810832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.810892 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.811296 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.811808 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.812888 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.813074 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.813134 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.814261 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.814262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.814262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.814269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.815835 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.816235 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.816236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.816236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.816297 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.816513 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.816513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.816514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.816614 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.816662 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.816830 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.816890 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:36.817016 1146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.810509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37538)
0929 19:05:36.810668 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:36.810680 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:36.810756 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:36.810911 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:36.810911 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:36.811120 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:36.811186 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.811188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.811896 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.811897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.812793 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:36.812796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.813157 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.813158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:36.813226 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.813738 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:36.813744 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:36.814169 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:36.815926 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.815927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.815928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.816154 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.816381 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:36.816382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:36.816383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:36.816450 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:36.816678 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:36.816679 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:36.816727 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 19:05:36.816799 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:05:36.816905 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260929 19:05:36.817585 595 tablet_copy_service.cc:140] P 6abeb109cf0d4a4da444f9e29f8fd826: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7842c1473cb44de6b63e14ddc22c71e6 ({username='oryx', principal='oryx/127.26.78.253@KRBTEST.COM'} at 127.0.0.1:37538)
I20260929 19:05:36.817696 595 tablet_copy_service.cc:161] P 6abeb109cf0d4a4da444f9e29f8fd826: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7842c1473cb44de6b63e14ddc22c71e6 at {username='oryx', principal='oryx/127.26.78.253@KRBTEST.COM'} at 127.0.0.1:37538: session id = 7842c1473cb44de6b63e14ddc22c71e6-00000000000000000000000000000000
I20260929 19:05:36.818593 595 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 19:05:36.819245 1040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 19:05:36.820340 1040 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: tablet copy: Starting download of 0 data blocks...
I20260929 19:05:36.820487 1040 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: tablet copy: Starting download of 1 WAL segments...
I20260929 19:05:36.821345 1040 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 19:05:36.822264 595 tablet_copy_service.cc:342] P 6abeb109cf0d4a4da444f9e29f8fd826: Request end of tablet copy session 7842c1473cb44de6b63e14ddc22c71e6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.78.253@KRBTEST.COM'} at 127.0.0.1:37538
I20260929 19:05:36.822335 595 tablet_copy_service.cc:433] P 6abeb109cf0d4a4da444f9e29f8fd826: ending tablet copy session 7842c1473cb44de6b63e14ddc22c71e6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7842c1473cb44de6b63e14ddc22c71e6
I20260929 19:05:36.825170 1040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:36.826501 1147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.826599 1148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.826668 1150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.826858 1040 server_base.cc:1034] running on GCE node
I20260929 19:05:36.826946 1040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:36.827076 1040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:36.828212 1040 hybrid_clock.cc:648] HybridClock initialized: now 1790708736828211 us; error 38 us; skew 500 ppm
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:36.829586 1040 init.cc:377] Logged in from keytab as oryx/127.26.78.253@KRBTEST.COM (short username oryx)
I20260929 19:05:36.829947 1040 webserver.cc:460] Webserver started at http://127.26.78.253:34201/ using document root <none> and password file <none>
I20260929 19:05:36.830081 1040 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.830157 1040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.830793 1040 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:36.831324 1157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.831458 1040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:36.831506 1040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
uuid: "7842c1473cb44de6b63e14ddc22c71e6"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "e160d0074c82b511ce0704989c7b1332"
server_key_iv: "c22eb60119f218b564fff3e672dbc9e4"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.831566 1040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:36.842229 1040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:36.843911 1040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:36.847005 1040 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.253:36035
I20260929 19:05:36.847218 1209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.253:36035 every 8 connection(s)
I20260929 19:05:36.847394 1040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
I20260929 19:05:36.848326 1210 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:05:36.848873 1210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Bootstrap starting.
I20260929 19:05:36.851889 1210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:05:36.852133 1210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: Bootstrap complete.
I20260929 19:05:36.852367 1210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } }
I20260929 19:05:36.852460 1210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7842c1473cb44de6b63e14ddc22c71e6, State: Initialized, Role: LEARNER
I20260929 19:05:36.852500 1210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } }
I20260929 19:05:36.852682 1210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:36.852701 1213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } }
I20260929 19:05:36.852819 1213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: This master's current role is: LEARNER
I20260929 19:05:36.854290 1225 catalog_manager.cc:1259] Loaded cluster ID: 0cbae24eb7c14371b06008a758666cf6
I20260929 19:05:36.854334 1225 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: loading cluster ID for follower catalog manager: success
I20260929 19:05:36.855846 1225 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: acquiring CA information for follower catalog manager: success
I20260929 19:05:36.856168 1225 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:05:36.861286 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:36.853717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.26.78.253:36035)
0929 19:05:36.853878 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:36.853880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:36.853887 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:05:36.853911 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:36.853912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:36.853923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:36.854870 (+ 947us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:36.854916 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.855976 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.856827 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:36.857122 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:36.857197 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:36.858495 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:36.858496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:36.858497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:36.858504 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:36.860072 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.860485 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.860487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.860487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.860544 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:36.860765 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:36.860766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:36.860766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:36.860880 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:36.860982 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:36.861096 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 19:05:36.861185 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for oryx/127.26.78.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.252.keytab.
Entry for principal oryx/127.26.78.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.252.keytab.
I20260929 19:05:36.889881 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:44833
--webserver_interface=127.26.78.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.252.keytab
--principal=oryx/127.26.78.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:05:36.972230 1228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:36.972441 1228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:36.972464 1228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:36.973560 1228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:05:36.973599 1228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:36.973613 1228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:36.973626 1228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:05:36.973645 1228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:05:36.975185 1228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.252.keytab
--principal=oryx/127.26.78.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:44833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.78.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:36.975384 1228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:36.975559 1228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:36.980227 1233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.980223 1236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:36.980265 1234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:36.980542 1228 server_base.cc:1034] running on GCE node
I20260929 19:05:36.980741 1228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:36.980975 1228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:36.982136 1228 hybrid_clock.cc:648] HybridClock initialized: now 1790708736982120 us; error 29 us; skew 500 ppm
Sep 29 19:05:36 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:36.984966 1228 init.cc:377] Logged in from keytab as oryx/127.26.78.252@KRBTEST.COM (short username oryx)
I20260929 19:05:36.985364 1228 webserver.cc:460] Webserver started at http://127.26.78.252:34797/ using document root <none> and password file <none>
I20260929 19:05:36.985560 1228 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:36.985600 1228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:36.985741 1228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:05:36.986832 1228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/instance:
uuid: "8d8f747ed6c9437aac8d74c8cf770758"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "5c51709ee6f74ba1fedce79cc9db068b"
server_key_iv: "d6daf60a376994d4720ecbff483b6974"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.987145 1228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal/instance:
uuid: "8d8f747ed6c9437aac8d74c8cf770758"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "5c51709ee6f74ba1fedce79cc9db068b"
server_key_iv: "d6daf60a376994d4720ecbff483b6974"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.988396 1228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:05:36.989111 1243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:36.989297 1228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:36.989378 1228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
uuid: "8d8f747ed6c9437aac8d74c8cf770758"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "5c51709ee6f74ba1fedce79cc9db068b"
server_key_iv: "d6daf60a376994d4720ecbff483b6974"
server_key_version: "encryptionkey@0"
I20260929 19:05:36.989442 1228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:37.002712 1228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:37.004998 1228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:37.005110 1228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:37.009199 1228 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.252:44833
I20260929 19:05:37.009274 1295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.252:44833 every 8 connection(s)
I20260929 19:05:37.009629 1228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
I20260929 19:05:37.010605 1296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:37.012218 1296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:05:37.014801 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 1228
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.252@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.78.252@KRBTEST.COM
I20260929 19:05:37.022922 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.013060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.26.78.254:44725)
0929 19:05:37.013120 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.013122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.013282 (+ 160us) server_negotiation.cc:407] Connection header received
0929 19:05:37.013795 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.013796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.013808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.013922 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.014582 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.015130 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.016662 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.016874 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.016969 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.019533 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.019535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.019536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.019544 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.021271 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.021861 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.021862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.021863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.021925 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.022229 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.022230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.022230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.022384 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.022459 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.022739 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.022818 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:05:37.023111 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.012924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37542)
0929 19:05:37.013147 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.013189 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.013672 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.014004 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.014007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.014303 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.014502 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.014506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.015205 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.015206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.016551 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.016554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.016998 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.017000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.017171 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.017758 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.017768 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.019422 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.021382 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.021390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.021392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.021756 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.022004 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.022005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.022005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.022114 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.022465 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.022470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.022596 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.022704 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.022857 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":145,"threads_started":1}
I20260929 19:05:37.023779 1296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:05:37.024837 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.015367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.26.78.252:44833)
0929 19:05:37.015516 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.015521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.015540 (+ 19us) server_negotiation.cc:407] Connection header received
0929 19:05:37.015558 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.015559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.015575 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.015637 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.016425 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.017095 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.018085 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.018412 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.018502 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.021286 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.021294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.021295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.021302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.023214 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.023680 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.023683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.023683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.023740 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.023952 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.023953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.023953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.024084 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.024658 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.024701 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.024765 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":96,"threads_started":1}
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.252@KRBTEST.COM for oryx/127.26.78.253@KRBTEST.COM
I20260929 19:05:37.030579 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.024075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.0.0.1:38398)
0929 19:05:37.024137 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.024148 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.024185 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.024378 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.024379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.024528 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.024582 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.024584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.025386 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.025387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.025975 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.025978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.026366 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.026367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.026416 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.026833 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.026837 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.027942 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.029618 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.029619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.029619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.029786 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.030078 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.030079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.030079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.030134 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.030356 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.030356 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.030393 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.030446 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.030519 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 19:05:37.030681 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.024161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.26.78.253:36035)
0929 19:05:37.024229 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.024231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.024241 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:37.024272 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.024273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.024283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.024374 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.024720 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.025309 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.026075 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.026296 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.026362 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.028018 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.028020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.028020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.028027 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.029534 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.029950 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.029951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.029951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.030012 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.030193 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.030194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.030194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.030298 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.030352 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.030463 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.030539 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:05:37.031128 1296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 44833 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.252@KRBTEST.COM for oryx/127.26.78.252@KRBTEST.COM
I20260929 19:05:37.037238 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.031498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.26.78.252:44833)
0929 19:05:37.031536 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.031538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.031579 (+ 41us) server_negotiation.cc:407] Connection header received
0929 19:05:37.031647 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.031647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.031657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.031710 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.031962 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.032486 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.033130 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.033329 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.033391 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.035006 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.035008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.035008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.035014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.036387 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.036656 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.036656 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.036657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.036698 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.036840 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.036840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.036841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.036928 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.036972 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.037073 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.037170 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":19}
I20260929 19:05:37.037250 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.031469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:44833 (local address 127.0.0.1:55204)
0929 19:05:37.031534 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.031545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.031594 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.031713 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.031713 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.031850 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.031878 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.031879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.032586 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.032587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.033060 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.033061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.033404 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.033405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.033440 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.033783 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.033786 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.034927 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.036457 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.036458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.036458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.036599 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.036751 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.036752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.036752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.036804 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.036978 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.036979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.037010 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.037059 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.037130 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 19:05:37.038785 1296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Bootstrap starting.
I20260929 19:05:37.039696 1296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:37.039988 1296 log.cc:824] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:37.040776 1296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: No bootstrap required, opened a new log
I20260929 19:05:37.042309 1296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 44833 } }
I20260929 19:05:37.042438 1296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:37.042460 1296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8f747ed6c9437aac8d74c8cf770758, State: Initialized, Role: FOLLOWER
I20260929 19:05:37.042582 1296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 44833 } }
I20260929 19:05:37.042831 1301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 44833 } } }
I20260929 19:05:37.042907 1296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:37.042917 1301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:05:37.044584 1312 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:05:37.044663 1312 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:05:37.047052 1228 master_runner.cc:428] Detected that this master 8d8f747ed6c9437aac8d74c8cf770758 is joining an existing cluster
I20260929 19:05:37.047092 1228 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.78.252:44833
I20260929 19:05:37.055404 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.048297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.26.78.252:44833)
0929 19:05:37.048331 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.048333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.048609 (+ 276us) server_negotiation.cc:407] Connection header received
0929 19:05:37.048696 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.048697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.048706 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.048732 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.049131 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.049859 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051085 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.051367 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051726 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.052807 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.052808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.052808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.052814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.054259 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.054639 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.054640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.054640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.054728 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.054968 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.054968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.054969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.055063 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.055114 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.055278 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.055343 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:05:37.055691 1320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.048131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:44833 (local address 127.0.0.1:55206)
0929 19:05:37.048545 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.048556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.048631 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.048785 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.048785 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.048994 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.049053 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.049054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.049944 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.049945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.050994 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.050997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.051810 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051810 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.051886 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.052251 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.052255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.052715 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.054340 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.054341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.054342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.054562 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.054808 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.054809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.054809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.054902 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.055135 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.055136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.055181 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.055264 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.055411 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":48,"thread_start_us":92,"threads_started":1}
I20260929 19:05:37.076749 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 637
I20260929 19:05:37.081729 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:40115
--local_ip_for_outbound_sockets=127.26.78.193
--tserver_master_addrs=127.26.78.254:44725
--webserver_port=34745
--webserver_interface=127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab
--principal=oryx/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:05:37.091583 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.048347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.26.78.253:36035)
0929 19:05:37.048400 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.048402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.048958 (+ 556us) server_negotiation.cc:407] Connection header received
0929 19:05:37.049044 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.049045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.049055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.049084 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.049458 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.050214 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051250 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.051524 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051558 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.087465 (+ 35907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.087467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.087467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.087476 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.090068 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.090501 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.090502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.090503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.090561 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.090936 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.090949 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.090949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.091102 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.091162 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.091391 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.091481 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:05:37.092463 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.048409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.0.0.1:38408)
0929 19:05:37.048909 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.048918 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.048980 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.049139 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.049140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.049327 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.049377 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.049379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.050305 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.050306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.051164 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.051167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.051654 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.051737 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.086937 (+ 35200us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.086942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.087328 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.090185 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.090186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.090187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.090411 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.090722 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.090730 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.090730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.090859 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.091217 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.091218 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.091267 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.091746 (+ 479us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.092345 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":74,"threads_started":1}
I20260929 19:05:37.093335 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.048080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37552)
0929 19:05:37.048252 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.048263 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.048342 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.048538 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.048539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.048707 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.048767 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.048768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.049734 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.049735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.050642 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.050644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.051181 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.051244 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.051762 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.051767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.086180 (+ 34413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.091247 (+ 5067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.091249 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.091250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.091724 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.092221 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.092221 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.092222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.092332 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.092874 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.092875 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.092931 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.093031 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.093162 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260929 19:05:37.093858 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.048149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37552 (local address 127.26.78.254:44725)
0929 19:05:37.048193 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.048195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.048315 (+ 120us) server_negotiation.cc:407] Connection header received
0929 19:05:37.048420 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.048422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.048435 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.048469 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.048861 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.049649 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.050768 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.051069 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.051126 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.086319 (+ 35193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.086321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.086322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.086334 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.091041 (+ 4707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.091953 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.091953 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.091954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.092056 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.092452 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.092453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.092453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.092709 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.093502 (+ 793us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.093554 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.093697 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708736, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.254@KRBTEST.COM for oryx/127.26.78.252@KRBTEST.COM
I20260929 19:05:37.107987 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.099563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:44833 (local address 127.0.0.1:55216)
0929 19:05:37.099615 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.099628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.099678 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.099955 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.099956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.100140 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.100194 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.100196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.101227 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.101228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.102115 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.102119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.102648 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.102649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.102727 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.103154 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.103161 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.104882 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.106865 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.106866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.106866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.107103 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.107336 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.107336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.107337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.107400 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.107684 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.107685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.107728 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.107792 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.107892 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 19:05:37.108659 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.099679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.26.78.252:44833)
0929 19:05:37.099732 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.099733 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.099743 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:37.099782 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.099783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.099795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.099869 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.100289 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.101101 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.102216 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.102526 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.102592 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.104983 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.104985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.104985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.104993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.106764 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.107185 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.107187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.107187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.107244 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.107471 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.107472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.107473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.107598 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.107658 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.108507 (+ 849us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.108582 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:05:37.109479 575 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.78.252:44833: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758"
member_type: NON_VOTER
last_known_addr {
host: "127.26.78.252"
port: 44833
}
attrs {
promote: true
}
}
dest_uuid: "6abeb109cf0d4a4da444f9e29f8fd826"
cas_config_opid_index: 5
I20260929 19:05:37.109766 575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:37.110190 1136 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8d8f747ed6c9437aac8d74c8cf770758 (127.26.78.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.111304 1136 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.78.252:44833
I20260929 19:05:37.111405 1136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.111485 1136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:37.112228 1270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:37.113230 1270 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 FOLLOWER]: Refusing update from remote peer 6abeb109cf0d4a4da444f9e29f8fd826: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 19:05:37.113523 1135 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:37.114183 1228 master.cc:537] Master@127.26.78.252:44833 shutting down...
I20260929 19:05:37.115631 1270 raft_consensus.cc:858] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 44833 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
W20260929 19:05:37.115693 1270 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 44833 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:37.117151 1270 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7842c1473cb44de6b63e14ddc22c71e6 (127.26.78.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } }
I20260929 19:05:37.118355 1301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6abeb109cf0d4a4da444f9e29f8fd826. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } }
I20260929 19:05:37.118489 1301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 19:05:37.118453 1135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.118531 1135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:37.118525 1301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } }
I20260929 19:05:37.118571 1301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 19:05:37.118628 1324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } }
I20260929 19:05:37.118692 1324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260929 19:05:37.119138 559 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 -> Peer 8d8f747ed6c9437aac8d74c8cf770758 (127.26.78.252:44833): Couldn't send request to peer 8d8f747ed6c9437aac8d74c8cf770758. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 19:05:37.120564 1228 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260929 19:05:37.120677 1228 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260929 19:05:37.120716 1228 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: stopping tablet replica
I20260929 19:05:37.122862 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.111793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.26.78.253:36035)
0929 19:05:37.111837 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.111839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.116432 (+ 4593us) server_negotiation.cc:407] Connection header received
0929 19:05:37.116542 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.116543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.116568 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.116608 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:37.116805 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.117576 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.121137 (+ 3561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.121844 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.121897 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.121982 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.122732 (+ 750us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.122785 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:05:37.123236 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.111693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.0.0.1:38424)
0929 19:05:37.116363 (+ 4670us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.116376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.116463 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.116688 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.116689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.116691 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:37.116737 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.116739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.117837 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.117838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.120280 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.120283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.122505 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.122506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.122582 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.122611 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.123118 (+ 507us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.123169 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4647}
I20260929 19:05:37.123868 1184 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 LEARNER]: Refusing update from remote peer 6abeb109cf0d4a4da444f9e29f8fd826: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 19:05:37.124101 1136 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 19:05:37.124702 1184 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8d8f747ed6c9437aac8d74c8cf770758 (127.26.78.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.126015 1213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6abeb109cf0d4a4da444f9e29f8fd826. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.126111 1213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: This master's current role is: LEARNER
I20260929 19:05:37.126147 1213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: NON_VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: true } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.126199 1213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: This master's current role is: LEARNER
I20260929 19:05:37.126550 1136 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: attempting to promote NON_VOTER 7842c1473cb44de6b63e14ddc22c71e6 to VOTER
I20260929 19:05:37.126763 1136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
W20260929 19:05:37.127645 559 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 -> Peer 8d8f747ed6c9437aac8d74c8cf770758 (127.26.78.252:44833): Couldn't send request to peer 8d8f747ed6c9437aac8d74c8cf770758. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 19:05:37.127825 1184 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 LEARNER]: Refusing update from remote peer 6abeb109cf0d4a4da444f9e29f8fd826: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 19:05:37.128010 1135 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 19:05:37.128958 1135 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7842c1473cb44de6b63e14ddc22c71e6 (127.26.78.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.130025 1135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.130103 1135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:37.130167 1135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.130215 1135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826 [sys.catalog]: This master's current role is: LEADER
I20260929 19:05:37.130618 1184 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7842c1473cb44de6b63e14ddc22c71e6 (127.26.78.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.131803 1330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.131896 1330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:05:37.145622 1228 master.cc:552] Master@127.26.78.252:44833 shutdown complete.
I20260929 19:05:37.145694 1228 master_runner.cc:305] Clearing existing system tablet
I20260929 19:05:37.145922 1228 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:37.145982 1228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:37.146852 1228 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:05:37.147423 1333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.147574 1228 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:37.147639 1228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:05:37.147667 1228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
uuid: "8d8f747ed6c9437aac8d74c8cf770758"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "5c51709ee6f74ba1fedce79cc9db068b"
server_key_iv: "d6daf60a376994d4720ecbff483b6974"
server_key_version: "encryptionkey@0"
I20260929 19:05:37.148540 1228 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:37.150332 1228 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 19:05:37.150427 1228 log.cc:1196] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 19:05:37.150650 1228 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Deleting consensus metadata
I20260929 19:05:37.150825 1228 master_runner.cc:315] Copying system tablet from 127.26.78.254:44725
I20260929 19:05:37.151603 1228 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: tablet copy: Beginning tablet copy session from remote peer at address 127.26.78.254:44725
I20260929 19:05:37.160276 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.151998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.26.78.254:44725)
0929 19:05:37.152046 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.152049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.154184 (+ 2135us) server_negotiation.cc:407] Connection header received
0929 19:05:37.154256 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.154257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.154271 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.154323 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.154708 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.155344 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.156326 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.156526 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.156624 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.157657 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.157658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.157659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.157666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.159122 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.159494 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.159495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.159496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.159550 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.159755 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.159756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.159756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.159870 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.159945 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.160101 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.160178 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:05:37.160298 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.151954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37564)
0929 19:05:37.154088 (+ 2134us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.154099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.154198 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.154379 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.154380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.154572 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.154621 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.154623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.155427 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.155428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.156227 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.156230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.156613 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.156614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.156676 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.157223 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.157228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.157533 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.159227 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.159228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.159229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.159406 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.159624 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.159625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.159626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.159684 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.159946 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.159947 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.159992 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.160076 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.160179 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2086,"thread_start_us":67,"threads_started":1}
I20260929 19:05:37.160905 595 tablet_copy_service.cc:140] P 6abeb109cf0d4a4da444f9e29f8fd826: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8d8f747ed6c9437aac8d74c8cf770758 ({username='oryx', principal='oryx/127.26.78.252@KRBTEST.COM'} at 127.0.0.1:37564)
I20260929 19:05:37.160990 595 tablet_copy_service.cc:161] P 6abeb109cf0d4a4da444f9e29f8fd826: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8d8f747ed6c9437aac8d74c8cf770758 at {username='oryx', principal='oryx/127.26.78.252@KRBTEST.COM'} at 127.0.0.1:37564: session id = 8d8f747ed6c9437aac8d74c8cf770758-00000000000000000000000000000000
I20260929 19:05:37.161454 595 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 19:05:37.162005 1228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 19:05:37.162984 1228 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: tablet copy: Starting download of 0 data blocks...
I20260929 19:05:37.163151 1228 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: tablet copy: Starting download of 1 WAL segments...
I20260929 19:05:37.163863 1228 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 19:05:37.164600 595 tablet_copy_service.cc:342] P 6abeb109cf0d4a4da444f9e29f8fd826: Request end of tablet copy session 8d8f747ed6c9437aac8d74c8cf770758-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.78.252@KRBTEST.COM'} at 127.0.0.1:37564
I20260929 19:05:37.164654 595 tablet_copy_service.cc:433] P 6abeb109cf0d4a4da444f9e29f8fd826: ending tablet copy session 8d8f747ed6c9437aac8d74c8cf770758-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8d8f747ed6c9437aac8d74c8cf770758
I20260929 19:05:37.167245 1228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:37.168759 1340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:37.169046 1343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:37.169240 1228 server_base.cc:1034] running on GCE node
W20260929 19:05:37.169937 1341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:37.170151 1228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:37.170317 1228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:37.171443 1228 hybrid_clock.cc:648] HybridClock initialized: now 1790708737171436 us; error 26 us; skew 500 ppm
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:37.172783 1228 init.cc:377] Logged in from keytab as oryx/127.26.78.252@KRBTEST.COM (short username oryx)
I20260929 19:05:37.173087 1228 webserver.cc:460] Webserver started at http://127.26.78.252:40937/ using document root <none> and password file <none>
I20260929 19:05:37.173206 1228 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:37.173243 1228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:37.173938 1228 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.174466 1350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.174615 1228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.174661 1228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
uuid: "8d8f747ed6c9437aac8d74c8cf770758"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "5c51709ee6f74ba1fedce79cc9db068b"
server_key_iv: "d6daf60a376994d4720ecbff483b6974"
server_key_version: "encryptionkey@0"
I20260929 19:05:37.174724 1228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:37.182878 1228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:37.185247 1228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:37.188537 1228 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.252:44833
I20260929 19:05:37.188560 1402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.252:44833 every 8 connection(s)
I20260929 19:05:37.188868 1228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
I20260929 19:05:37.189632 1403 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:05:37.190157 1403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Bootstrap starting.
I20260929 19:05:37.191983 1403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:05:37.192227 1403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: Bootstrap complete.
I20260929 19:05:37.192428 1403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:37.192519 1403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8d8f747ed6c9437aac8d74c8cf770758, State: Initialized, Role: LEARNER
I20260929 19:05:37.192564 1403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:37.192767 1403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:05:37.192729 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } } }
I20260929 19:05:37.192950 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758 [sys.catalog]: This master's current role is: LEARNER
I20260929 19:05:37.194232 1417 catalog_manager.cc:1259] Loaded cluster ID: 0cbae24eb7c14371b06008a758666cf6
I20260929 19:05:37.194326 1417 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: loading cluster ID for follower catalog manager: success
I20260929 19:05:37.195402 1417 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: acquiring CA information for follower catalog manager: success
I20260929 19:05:37.195688 1417 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d8f747ed6c9437aac8d74c8cf770758: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 19:05:37.213327 1321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:37.213613 1321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:37.213647 1321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:37.215488 1321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:37.215555 1321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:37.215598 1321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:05:37.217979 1321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab
--principal=oryx/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:40115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=34745
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:37.218241 1321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:37.218489 1321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:37.224001 1422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:37.223981 1425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:37.223981 1423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:37.224254 1321 server_base.cc:1034] running on GCE node
I20260929 19:05:37.224545 1321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:37.224818 1321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:37.225998 1321 hybrid_clock.cc:648] HybridClock initialized: now 1790708737225984 us; error 30 us; skew 500 ppm
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:37.228570 1321 init.cc:377] Logged in from keytab as oryx/127.26.78.193@KRBTEST.COM (short username oryx)
I20260929 19:05:37.229031 1321 webserver.cc:460] Webserver started at http://127.26.78.193:34745/ using document root <none> and password file <none>
I20260929 19:05:37.229269 1321 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:37.229358 1321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:37.230788 1321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:37.231473 1432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.231639 1321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:37.231742 1321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "1663f4cc14bf4d49b8f09597b73caf0b"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "f8ce6d371f8411bb93c61f5544d142b1"
server_key_iv: "10f86c0b5829575288dbd0e911c370fd"
server_key_version: "encryptionkey@0"
I20260929 19:05:37.232137 1321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:37.245450 1321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:37.247999 1321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:37.248179 1321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:37.248430 1321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:37.248749 1321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:37.248780 1321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.248840 1321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:37.248865 1321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.255309 1321 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:40115
I20260929 19:05:37.255411 1546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:40115 every 8 connection(s)
I20260929 19:05:37.255744 1321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:05:37.258286 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 1321
I20260929 19:05:37.258474 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 771
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:05:37.264389 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:33085
--local_ip_for_outbound_sockets=127.26.78.194
--tserver_master_addrs=127.26.78.254:44725
--webserver_port=41421
--webserver_interface=127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab
--principal=oryx/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.253@KRBTEST.COM
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.252@KRBTEST.COM
I20260929 19:05:37.268832 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.256530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:58481 (local address 127.26.78.254:44725)
0929 19:05:37.256573 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.256576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.256986 (+ 410us) server_negotiation.cc:407] Connection header received
0929 19:05:37.257549 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.257550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.257562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.257595 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.258551 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.259302 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.261539 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.261907 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.261972 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.265311 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.265313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.265313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.265321 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.267119 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.267699 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.267701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.267701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.267762 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.268038 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.268039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.268039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.268182 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.268225 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.268629 (+ 404us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.268722 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:05:37.269007 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.256820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.26.78.193:39483)
0929 19:05:37.257063 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.257071 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.257452 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.258086 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.258087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.258242 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.258442 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.258449 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.259852 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.259852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.260922 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.260924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.261486 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.261487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.261586 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.263205 (+ 1619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.263210 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.265192 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.267354 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.267355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.267355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.267673 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.267895 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.267896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.267896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.267987 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.268324 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.268325 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.268467 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.268578 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.268763 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":9,"thread_start_us":57,"threads_started":1}
I20260929 19:05:37.269235 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.256521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:58481)
0929 19:05:37.256916 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.256926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.257449 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.257710 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.257711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.258128 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.258439 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.258445 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.259610 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.259611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.260922 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.260924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.262025 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.262027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.262106 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.262521 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.262531 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.265193 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.267218 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.267226 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.267228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.267613 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.267830 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.267831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.267832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.267967 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.268269 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.268275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.268467 (+ 192us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.268962 (+ 495us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.269086 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":34,"thread_start_us":90,"threads_started":1}
I20260929 19:05:37.270120 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.256992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:39483 (local address 127.26.78.253:36035)
0929 19:05:37.257039 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.257041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.257141 (+ 100us) server_negotiation.cc:407] Connection header received
0929 19:05:37.257549 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.257550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.257562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.257595 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.259037 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.259765 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.261021 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.261398 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.261477 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.265313 (+ 3836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.265315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.265315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.265324 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.267273 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.267757 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.267758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.267759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.267820 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.268063 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.268064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.268064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.268216 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.268282 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.269909 (+ 1627us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.270015 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:05:37.270157 1547 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44725
I20260929 19:05:37.270252 1547 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.270447 1547 heartbeater.cc:511] Master 127.26.78.254:44725 requested a full tablet report, sending...
I20260929 19:05:37.270449 1549 heartbeater.cc:347] Connected to a master server at 127.26.78.253:36035
I20260929 19:05:37.270537 1549 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.270620 1549 heartbeater.cc:511] Master 127.26.78.253:36035 requested a full tablet report, sending...
I20260929 19:05:37.270957 575 ts_manager.cc:194] Re-registered known tserver with Master: 1663f4cc14bf4d49b8f09597b73caf0b (127.26.78.193:40115)
I20260929 19:05:37.270969 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.256593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:37853 (local address 127.26.78.252:44833)
0929 19:05:37.256999 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.257001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.257010 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:37.257537 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.257538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.257553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.257619 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.262354 (+ 4735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.263036 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.264102 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.264330 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.264813 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.267421 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.267423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.267423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.267431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.269189 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.269587 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.269588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.269589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.269641 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.269861 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.269862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.269862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.269996 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.270039 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.270777 (+ 738us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.270884 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":58,"threads_started":1}
I20260929 19:05:37.271032 1174 ts_manager.cc:194] Registered new tserver with Master: 1663f4cc14bf4d49b8f09597b73caf0b (127.26.78.193:40115)
I20260929 19:05:37.271443 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.256522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:44833 (local address 127.26.78.193:37853)
0929 19:05:37.256696 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.256748 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.257428 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.257689 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.257693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.258145 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.258440 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.258447 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.263146 (+ 4699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.263147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.263909 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.263912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.264430 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.264431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.264519 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.264995 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.265000 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.267290 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.269279 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.269280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.269281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.269516 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.269703 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.269704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.269704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.269751 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.270089 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.270090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.270644 (+ 554us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.271070 (+ 426us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.271168 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260929 19:05:37.271476 575 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:58481
I20260929 19:05:37.271788 1548 heartbeater.cc:347] Connected to a master server at 127.26.78.252:44833
I20260929 19:05:37.271845 1548 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.271920 1548 heartbeater.cc:511] Master 127.26.78.252:44833 requested a full tablet report, sending...
I20260929 19:05:37.272339 1362 ts_manager.cc:194] Registered new tserver with Master: 1663f4cc14bf4d49b8f09597b73caf0b (127.26.78.193:40115)
W20260929 19:05:37.350669 1555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:37.350884 1555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:37.350908 1555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:37.352005 1555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:37.352038 1555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:37.352063 1555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:05:37.353693 1555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab
--principal=oryx/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:33085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=41421
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:37.354007 1555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:37.354296 1555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:37.359526 1560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:37.359542 1555 server_base.cc:1034] running on GCE node
W20260929 19:05:37.359557 1561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:37.359526 1563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:37.359926 1555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:37.360231 1555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:37.361410 1555 hybrid_clock.cc:648] HybridClock initialized: now 1790708737361398 us; error 29 us; skew 500 ppm
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:37.364017 1555 init.cc:377] Logged in from keytab as oryx/127.26.78.194@KRBTEST.COM (short username oryx)
I20260929 19:05:37.364495 1555 webserver.cc:460] Webserver started at http://127.26.78.194:41421/ using document root <none> and password file <none>
I20260929 19:05:37.364745 1555 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:37.364830 1555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:37.366245 1555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:37.366870 1570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.367041 1555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:05:37.367147 1555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d86b1258564348998c1cc67688a4111d"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "74389f89eba68d7c93124d65bd89613b"
server_key_iv: "c841924cf48ff9ebca94c82ca70e1ad2"
server_key_version: "encryptionkey@0"
I20260929 19:05:37.367497 1555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:37.381460 1555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:37.383518 1555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:37.383677 1555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:37.383922 1555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:37.384234 1555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:37.384295 1555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.384328 1555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:37.384361 1555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.390198 1555 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:33085
I20260929 19:05:37.390235 1683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:33085 every 8 connection(s)
I20260929 19:05:37.390604 1555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.253@KRBTEST.COM
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.252@KRBTEST.COM
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:05:37.400732 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 1555
I20260929 19:05:37.400908 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 906
I20260929 19:05:37.403024 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.391733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:52135 (local address 127.26.78.254:44725)
0929 19:05:37.391784 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.391785 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.391958 (+ 173us) server_negotiation.cc:407] Connection header received
0929 19:05:37.392540 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.392541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.392553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.392629 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.394227 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.394988 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395970 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.396260 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.396327 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.399317 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.399318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.399318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.399327 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.401571 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.401959 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.401960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.401960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.402014 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.402226 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.402227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.402228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.402352 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.402425 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.402778 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.402879 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:37.403190 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.391646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:52135)
0929 19:05:37.391891 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.391899 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.392296 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.392677 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.392678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.393036 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.393339 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.393346 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.395081 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.395883 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.395885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.396343 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.396344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.396416 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.396940 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.396945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.399196 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.401638 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.401639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.401640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.401897 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.402092 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.402093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.402094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.402150 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.402428 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.402435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.402582 (+ 147us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.402754 (+ 172us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.402944 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260929 19:05:37.403270 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.391437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:39819 (local address 127.26.78.252:44833)
0929 19:05:37.391496 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.391499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.391843 (+ 344us) server_negotiation.cc:407] Connection header received
0929 19:05:37.392390 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.392392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.392410 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.392459 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.393435 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.394082 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395243 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.395489 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395568 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.398762 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.398764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.398764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.398772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.400135 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.400964 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.400965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.400966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.401023 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.402703 (+ 1680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.402704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.402704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.402835 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.402895 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.403073 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.403160 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:05:37.403644 1684 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44725
I20260929 19:05:37.403748 1684 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.403882 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.391343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:44833 (local address 127.26.78.194:39819)
0929 19:05:37.391759 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.391768 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.392285 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.392541 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.392546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.393016 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.393340 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.393346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.394211 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.394212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.395157 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.395159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.395603 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.395789 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.396464 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.396476 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.398652 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.400218 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.400224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.400227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.400878 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.402554 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.402555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.402555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.402624 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.402924 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.402924 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.402962 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.403629 (+ 667us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.403744 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":99,"thread_start_us":93,"threads_started":1}
I20260929 19:05:37.403965 1684 heartbeater.cc:511] Master 127.26.78.254:44725 requested a full tablet report, sending...
I20260929 19:05:37.404320 575 ts_manager.cc:194] Re-registered known tserver with Master: d86b1258564348998c1cc67688a4111d (127.26.78.194:33085)
I20260929 19:05:37.404865 575 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:52135
I20260929 19:05:37.406168 1685 heartbeater.cc:347] Connected to a master server at 127.26.78.252:44833
I20260929 19:05:37.406222 1685 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.406320 1685 heartbeater.cc:511] Master 127.26.78.252:44833 requested a full tablet report, sending...
I20260929 19:05:37.406599 1362 ts_manager.cc:194] Registered new tserver with Master: d86b1258564348998c1cc67688a4111d (127.26.78.194:33085)
I20260929 19:05:37.406772 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:41141
--local_ip_for_outbound_sockets=127.26.78.195
--tserver_master_addrs=127.26.78.254:44725
--webserver_port=40693
--webserver_interface=127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab
--principal=oryx/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:05:37.407095 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.391344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.26.78.194:37755)
0929 19:05:37.391564 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.391607 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.392285 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.392908 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.392908 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.393115 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.393340 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.393346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.394169 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.394170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.395237 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.395240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.395716 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.395789 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.396492 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.396496 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.398170 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.400279 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.400279 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.400280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.400623 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.400829 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.400830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.400830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.400974 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.406720 (+ 5746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.406721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.406811 (+ 90us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.406893 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.407019 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":44,"thread_start_us":83,"threads_started":1}
I20260929 19:05:37.407408 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.391368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:37755 (local address 127.26.78.253:36035)
0929 19:05:37.391445 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.391448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.391704 (+ 256us) server_negotiation.cc:407] Connection header received
0929 19:05:37.392459 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.392461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.392473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.392524 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.393451 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.394061 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395303 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.395621 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.395704 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.398272 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.398274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.398274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.398282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.400158 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.400689 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.400690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.400691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.400749 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.406439 (+ 5690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.406440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.406440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.406597 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.406673 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.407238 (+ 565us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.407322 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:37.407759 1686 heartbeater.cc:347] Connected to a master server at 127.26.78.253:36035
I20260929 19:05:37.407807 1686 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.407869 1686 heartbeater.cc:511] Master 127.26.78.253:36035 requested a full tablet report, sending...
I20260929 19:05:37.408134 1174 ts_manager.cc:194] Registered new tserver with Master: d86b1258564348998c1cc67688a4111d (127.26.78.194:33085)
W20260929 19:05:37.488216 1691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:05:37.488472 1691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:05:37.488514 1691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:05:37.489687 1691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:05:37.489758 1691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:05:37.489801 1691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:05:37.491297 1691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab
--principal=oryx/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:41141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=40693
--tserver_master_addrs=127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:05:37.491611 1691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:05:37.491860 1691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:05:37.497073 1699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:05:37.497120 1696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:37.497131 1691 server_base.cc:1034] running on GCE node
W20260929 19:05:37.497083 1697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:05:37.497501 1691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:05:37.497771 1691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:05:37.498958 1691 hybrid_clock.cc:648] HybridClock initialized: now 1790708737498946 us; error 26 us; skew 500 ppm
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:05:37.501492 1691 init.cc:377] Logged in from keytab as oryx/127.26.78.195@KRBTEST.COM (short username oryx)
I20260929 19:05:37.502036 1691 webserver.cc:460] Webserver started at http://127.26.78.195:40693/ using document root <none> and password file <none>
I20260929 19:05:37.502272 1691 fs_manager.cc:359] Metadata directory not provided
I20260929 19:05:37.502355 1691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:05:37.503707 1691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:05:37.504319 1706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.504520 1691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:05:37.504617 1691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "a526a417a8e146319a0e1b6ea63c9004"
format_stamp: "Formatted at 2026-09-29 19:05:36 on dist-test-slave-xz51"
server_key: "7713b0d1c39bf9cc045e99120f8dc380"
server_key_iv: "507ece0063ade1a4e43483c8a4e7ed59"
server_key_version: "encryptionkey@0"
I20260929 19:05:37.504978 1691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:05:37.516008 1691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:05:37.518169 1691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:05:37.518345 1691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:05:37.518597 1691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:05:37.518941 1691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:05:37.519007 1691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.519053 1691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:05:37.519079 1691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:05:37.525703 1691 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:41141
I20260929 19:05:37.525739 1819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:41141 every 8 connection(s)
I20260929 19:05:37.526155 1691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:05:37.531664 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 1691
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.253@KRBTEST.COM
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.252@KRBTEST.COM
Sep 29 19:05:37 dist-test-slave-xz51 krb5kdc[530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708737, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:05:37.537326 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.527341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:51365 (local address 127.26.78.253:36035)
0929 19:05:37.527388 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.527390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.527589 (+ 199us) server_negotiation.cc:407] Connection header received
0929 19:05:37.527910 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.527911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.527924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.527957 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.528739 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.529507 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.530644 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.530893 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.530988 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.533754 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.533755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.533756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.533764 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.535583 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.536168 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.536168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.536169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.536224 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.536535 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.536536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.536536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.536680 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.536724 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.537120 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.537197 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:05:37.537930 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.527225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36035 (local address 127.26.78.195:51365)
0929 19:05:37.527520 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.527526 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.527817 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.528080 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.528081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.528419 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.528625 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.528630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.529599 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.529600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.530554 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.530555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.530991 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.530992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.531058 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.531711 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.531716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.533615 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.535707 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.535714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.535716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.536087 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.536311 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.536311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.536312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.536450 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.536773 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.536779 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.536953 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.537467 (+ 514us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.537650 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":52,"thread_start_us":56,"threads_started":1}
I20260929 19:05:37.538129 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.526921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:44833 (local address 127.26.78.195:58595)
0929 19:05:37.527123 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.527162 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.527795 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.528059 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.528063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.528416 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.528624 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.528628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.529377 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.529378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.530291 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.530294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.530760 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.530761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.530908 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.531532 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.531545 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.534778 (+ 3233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.536824 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.536825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.536826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.537069 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.537311 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.537312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.537313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.537389 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.537732 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.537733 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.537776 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.537885 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.537998 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260929 19:05:37.538465 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.526928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:58595 (local address 127.26.78.252:44833)
0929 19:05:37.526981 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.526984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.527271 (+ 287us) server_negotiation.cc:407] Connection header received
0929 19:05:37.527910 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.527912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.527926 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.527987 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.528733 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.529244 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.530405 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.530651 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.530736 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.534911 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.534912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.534913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.534922 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.536730 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.537158 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.537159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.537160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.537230 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.537478 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.537479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.537480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.537630 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.537698 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.538279 (+ 581us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.538363 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:05:37.538714 1822 heartbeater.cc:347] Connected to a master server at 127.26.78.253:36035
I20260929 19:05:37.538822 1822 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.539040 1822 heartbeater.cc:511] Master 127.26.78.253:36035 requested a full tablet report, sending...
I20260929 19:05:37.539443 1821 heartbeater.cc:347] Connected to a master server at 127.26.78.252:44833
I20260929 19:05:37.539498 1821 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.539572 1821 heartbeater.cc:511] Master 127.26.78.252:44833 requested a full tablet report, sending...
I20260929 19:05:37.539724 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.526931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:43855 (local address 127.26.78.254:44725)
0929 19:05:37.526981 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.526983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.527472 (+ 489us) server_negotiation.cc:407] Connection header received
0929 19:05:37.527910 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.527912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.527924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.527957 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.528738 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.529500 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.530666 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.530949 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.531016 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.536703 (+ 5687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.536704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.536705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.536712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.538392 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.538793 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.538795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.538795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.538863 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.539118 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.539119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.539119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.539247 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.539293 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.539507 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.539594 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:05:37.539938 1362 ts_manager.cc:194] Registered new tserver with Master: a526a417a8e146319a0e1b6ea63c9004 (127.26.78.195:41141)
I20260929 19:05:37.539938 1174 ts_manager.cc:194] Registered new tserver with Master: a526a417a8e146319a0e1b6ea63c9004 (127.26.78.195:41141)
I20260929 19:05:37.540063 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.526921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:43855)
0929 19:05:37.527415 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.527424 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.527813 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.528436 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.528437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.528574 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:05:37.528625 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.528630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.529599 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.529600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.530550 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.530553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.531073 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.531074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.531141 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.532651 (+ 1510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:05:37.532656 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:05:37.535897 (+ 3241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:05:37.538473 (+ 2576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.538474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.538475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.538695 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.538961 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:05:37.538962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:05:37.538963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:05:37.539033 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:05:37.539353 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:05:37.539354 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:05:37.539393 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.539798 (+ 405us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.539909 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":82,"thread_start_us":77,"threads_started":1}
I20260929 19:05:37.540612 1820 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44725
I20260929 19:05:37.540673 1820 heartbeater.cc:464] Registering TS with master...
I20260929 19:05:37.540738 1820 heartbeater.cc:511] Master 127.26.78.254:44725 requested a full tablet report, sending...
I20260929 19:05:37.540964 575 ts_manager.cc:194] Re-registered known tserver with Master: a526a417a8e146319a0e1b6ea63c9004 (127.26.78.195:41141)
I20260929 19:05:37.541476 575 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:43855
I20260929 19:05:37.542836 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.532748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.26.78.252:44833)
0929 19:05:37.532991 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.532993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.533161 (+ 168us) server_negotiation.cc:407] Connection header received
0929 19:05:37.533183 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.533184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.533197 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.533253 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.534077 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.534865 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.535953 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.536270 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.536329 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.539422 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.539423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.539423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.539430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.541250 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.541668 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.541669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.541670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.541756 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.542025 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.542026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.542026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.542186 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.542292 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.542677 (+ 385us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.542761 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":43,"threads_started":1}
I20260929 19:05:37.546185 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.532760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.26.78.254:44725)
0929 19:05:37.533921 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.533923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.533931 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:37.533952 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.533953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.533965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.534573 (+ 608us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.536430 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.540605 (+ 4175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.541470 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.541759 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.541856 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.543320 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.543321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.543322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.543328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.544905 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.545348 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.545349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.545350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.545409 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.545617 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.545618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.545619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.545750 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.545908 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.546001 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.546080 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":81,"threads_started":1}
I20260929 19:05:37.548270 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.533342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.26.78.253:36035)
0929 19:05:37.533602 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.533605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.534518 (+ 913us) server_negotiation.cc:407] Connection header received
0929 19:05:37.536297 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.536299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.536313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.536377 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.541753 (+ 5376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.542491 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.543514 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.543856 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.543948 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.545251 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.545252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.545252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.545259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.547026 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.547431 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.547432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.547432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.547494 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.547721 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.547723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.547723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.547845 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.547957 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.548072 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.548164 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":70,"threads_started":1}
I20260929 19:05:37.547906 575 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:05:37.549669 575 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:05:37.557497 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.553533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:33085 (local address 127.0.0.1:39506)
0929 19:05:37.553646 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.553659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.553705 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.554087 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.554088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.554089 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:37.554131 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.554132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.555403 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.555405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.556390 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.556392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.557256 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.557257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.557316 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.557320 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.557370 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.557409 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":3}
I20260929 19:05:37.557745 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.553672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39506 (local address 127.26.78.194:33085)
0929 19:05:37.553890 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.553895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.553907 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:05:37.553949 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.553951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.553969 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.554089 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:37.554191 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.555311 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.556464 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.557173 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.557281 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.557372 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.557397 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.557463 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":104,"threads_started":1}
I20260929 19:05:37.557942 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.553643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:40115 (local address 127.0.0.1:49280)
0929 19:05:37.553978 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.553987 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.554027 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.554562 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.554562 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.554564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:37.554599 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.554600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.555464 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.555466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.556648 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.556651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.557645 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.557646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.557703 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.557707 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.557751 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.557789 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":62,"threads_started":1}
I20260929 19:05:37.558179 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.553889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.26.78.193:40115)
0929 19:05:37.554403 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.554407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.554415 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:05:37.554448 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.554451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.554469 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.554554 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:37.554651 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.555326 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.556731 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.557444 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.557534 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.557619 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.558045 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.558094 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":87,"threads_started":1}
I20260929 19:05:37.558571 1618 tablet_service.cc:1467] Processing CreateTablet for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f (DEFAULT_TABLE table=test-table [id=d6d1f8cbb76e44b38417944b5c31fc1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:37.558913 1618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:37.559104 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.553541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:41141 (local address 127.0.0.1:48932)
0929 19:05:37.553918 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.553929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.553981 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.554272 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.554273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.554274 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:37.554326 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.554328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.556741 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.556742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.557932 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.557934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.558860 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.558861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.558936 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.558940 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.558996 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.559034 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":11,"thread_start_us":86,"threads_started":1}
I20260929 19:05:37.559118 1844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.553808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.26.78.195:41141)
0929 19:05:37.554128 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.554132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.554141 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:37.554178 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.554180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.554197 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.554275 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:37.554412 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.555413 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.558037 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.558764 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.558848 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.558923 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.559009 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.559046 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":90,"threads_started":1}
I20260929 19:05:37.559774 1754 tablet_service.cc:1467] Processing CreateTablet for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f (DEFAULT_TABLE table=test-table [id=d6d1f8cbb76e44b38417944b5c31fc1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:37.560139 1754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:37.562667 1480 tablet_service.cc:1467] Processing CreateTablet for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f (DEFAULT_TABLE table=test-table [id=d6d1f8cbb76e44b38417944b5c31fc1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:05:37.562922 1846 tablet_bootstrap.cc:492] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Bootstrap starting.
I20260929 19:05:37.562989 1480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:05:37.563555 1846 tablet_bootstrap.cc:654] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:37.563582 1847 tablet_bootstrap.cc:492] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Bootstrap starting.
I20260929 19:05:37.563848 1846 log.cc:824] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:37.564155 1847 tablet_bootstrap.cc:654] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:37.564460 1847 log.cc:824] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:37.564777 1846 tablet_bootstrap.cc:492] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: No bootstrap required, opened a new log
I20260929 19:05:37.564920 1846 ts_tablet_manager.cc:1397] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:05:37.565182 1847 tablet_bootstrap.cc:492] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: No bootstrap required, opened a new log
I20260929 19:05:37.565225 1847 ts_tablet_manager.cc:1397] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:05:37.566504 1850 tablet_bootstrap.cc:492] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Bootstrap starting.
I20260929 19:05:37.567005 1846 raft_consensus.cc:348] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.567135 1846 raft_consensus.cc:374] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:37.567189 1846 raft_consensus.cc:729] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86b1258564348998c1cc67688a4111d, State: Initialized, Role: FOLLOWER
I20260929 19:05:37.567344 1850 tablet_bootstrap.cc:654] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Neither blocks nor log segments found. Creating new log.
I20260929 19:05:37.567335 1846 consensus_queue.cc:260] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.567602 1846 ts_tablet_manager.cc:1428] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:05:37.567678 1850 log.cc:824] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:05:37.567699 1684 heartbeater.cc:503] Master 127.26.78.254:44725 was elected leader, sending a full tablet report...
I20260929 19:05:37.567625 1847 raft_consensus.cc:348] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.567770 1847 raft_consensus.cc:374] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:37.567803 1847 raft_consensus.cc:729] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a526a417a8e146319a0e1b6ea63c9004, State: Initialized, Role: FOLLOWER
I20260929 19:05:37.567950 1847 consensus_queue.cc:260] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.568205 1847 ts_tablet_manager.cc:1428] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:05:37.568251 1820 heartbeater.cc:503] Master 127.26.78.254:44725 was elected leader, sending a full tablet report...
I20260929 19:05:37.569408 1850 tablet_bootstrap.cc:492] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: No bootstrap required, opened a new log
I20260929 19:05:37.569481 1850 ts_tablet_manager.cc:1397] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:05:37.571779 1850 raft_consensus.cc:348] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.571938 1850 raft_consensus.cc:374] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:05:37.571983 1850 raft_consensus.cc:729] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1663f4cc14bf4d49b8f09597b73caf0b, State: Initialized, Role: FOLLOWER
I20260929 19:05:37.572095 1850 consensus_queue.cc:260] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.572350 1850 ts_tablet_manager.cc:1428] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:05:37.572508 1547 heartbeater.cc:503] Master 127.26.78.254:44725 was elected leader, sending a full tablet report...
I20260929 19:05:37.606618 1853 raft_consensus.cc:482] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:05:37.606743 1853 raft_consensus.cc:504] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.607163 1853 leader_election.cc:288] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86b1258564348998c1cc67688a4111d (127.26.78.194:33085), 1663f4cc14bf4d49b8f09597b73caf0b (127.26.78.193:40115)
I20260929 19:05:37.610124 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.607476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:53409 (local address 127.26.78.194:33085)
0929 19:05:37.607527 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.607530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.607541 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:05:37.607588 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.607589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.607601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.607660 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:37.607852 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.608334 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.609368 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.609727 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.609800 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.609807 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.610002 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.610050 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:05:37.610165 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.607563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:36719 (local address 127.26.78.193:40115)
0929 19:05:37.607608 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.607609 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.607618 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:05:37.607643 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.607644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.607653 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.607707 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:05:37.607866 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.608327 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.609389 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.609749 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.609887 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.609895 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.610059 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.610101 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:05:37.610163 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.607350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:33085 (local address 127.26.78.195:53409)
0929 19:05:37.607428 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.607443 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.607504 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.607678 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.607679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.607681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:37.607761 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.607762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.608446 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.608447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.609267 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.609269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.609848 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.609849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.609926 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.609929 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.609983 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.610013 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 19:05:37.610174 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.607493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:40115 (local address 127.26.78.195:36719)
0929 19:05:37.607541 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:37.607552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:05:37.607589 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:05:37.607711 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:05:37.607711 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:05:37.607712 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:05:37.607770 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.607771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.608411 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.608413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.609268 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:05:37.609270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.609887 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.609888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:05:37.609978 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.609982 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:05:37.610040 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:05:37.610078 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 19:05:37.610558 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ec4592a4c34d008d9d22ebc8dd7d0f" candidate_uuid: "a526a417a8e146319a0e1b6ea63c9004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86b1258564348998c1cc67688a4111d" is_pre_election: true
I20260929 19:05:37.610656 1501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ec4592a4c34d008d9d22ebc8dd7d0f" candidate_uuid: "a526a417a8e146319a0e1b6ea63c9004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" is_pre_election: true
I20260929 19:05:37.610728 1638 raft_consensus.cc:2454] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a526a417a8e146319a0e1b6ea63c9004 in term 0.
I20260929 19:05:37.610769 1501 raft_consensus.cc:2454] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a526a417a8e146319a0e1b6ea63c9004 in term 0.
I20260929 19:05:37.610975 1708 leader_election.cc:302] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1663f4cc14bf4d49b8f09597b73caf0b, a526a417a8e146319a0e1b6ea63c9004; no voters:
I20260929 19:05:37.611199 1853 raft_consensus.cc:2781] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:05:37.611256 1853 raft_consensus.cc:482] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:05:37.611275 1853 raft_consensus.cc:3037] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:37.612041 1853 raft_consensus.cc:504] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.612183 1853 leader_election.cc:288] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [CANDIDATE]: Term 1 election: Requested vote from peers d86b1258564348998c1cc67688a4111d (127.26.78.194:33085), 1663f4cc14bf4d49b8f09597b73caf0b (127.26.78.193:40115)
I20260929 19:05:37.612408 1501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ec4592a4c34d008d9d22ebc8dd7d0f" candidate_uuid: "a526a417a8e146319a0e1b6ea63c9004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1663f4cc14bf4d49b8f09597b73caf0b"
I20260929 19:05:37.612488 1501 raft_consensus.cc:3037] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:37.612735 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ec4592a4c34d008d9d22ebc8dd7d0f" candidate_uuid: "a526a417a8e146319a0e1b6ea63c9004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86b1258564348998c1cc67688a4111d"
I20260929 19:05:37.612874 1638 raft_consensus.cc:3037] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:05:37.613354 1501 raft_consensus.cc:2454] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a526a417a8e146319a0e1b6ea63c9004 in term 1.
I20260929 19:05:37.613554 1708 leader_election.cc:302] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1663f4cc14bf4d49b8f09597b73caf0b, a526a417a8e146319a0e1b6ea63c9004; no voters:
I20260929 19:05:37.613646 1638 raft_consensus.cc:2454] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a526a417a8e146319a0e1b6ea63c9004 in term 1.
I20260929 19:05:37.613653 1853 raft_consensus.cc:2781] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:05:37.613855 1853 raft_consensus.cc:686] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 1 LEADER]: Becoming Leader. State: Replica: a526a417a8e146319a0e1b6ea63c9004, State: Running, Role: LEADER
I20260929 19:05:37.614053 1853 consensus_queue.cc:237] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } }
I20260929 19:05:37.615024 575 catalog_manager.cc:5495] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 reported cstate change: term changed from 0 to 1, leader changed from <none> to a526a417a8e146319a0e1b6ea63c9004 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "a526a417a8e146319a0e1b6ea63c9004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a526a417a8e146319a0e1b6ea63c9004" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 41141 } health_report { overall_health: HEALTHY } } }
I20260929 19:05:37.630707 1844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.628162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.26.78.195:41141)
0929 19:05:37.628227 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.628229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.628239 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:05:37.628276 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.628276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.628299 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.628383 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:05:37.628530 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.629026 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.629888 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.630106 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.630192 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.630377 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:05:37.630477 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:05:37.630580 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.630598 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.630619 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:05:37.632735 1638 raft_consensus.cc:1261] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 1 FOLLOWER]: Refusing update from remote peer a526a417a8e146319a0e1b6ea63c9004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:37.632826 1501 raft_consensus.cc:1261] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 1 FOLLOWER]: Refusing update from remote peer a526a417a8e146319a0e1b6ea63c9004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:05:37.633060 1853 consensus_queue.cc:1035] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86b1258564348998c1cc67688a4111d" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 33085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:37.633225 1855 consensus_queue.cc:1035] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1663f4cc14bf4d49b8f09597b73caf0b" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 40115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:05:37.637074 1859 mvcc.cc:205] Tried to move back new op lower bound from 7334742989340844032 to 7334742989268086784. Current Snapshot: MvccSnapshot[applied={T|T < 7334742989340844032 or (T in {7334742989340844032})}]
I20260929 19:05:37.637229 1860 mvcc.cc:205] Tried to move back new op lower bound from 7334742989340844032 to 7334742989268086784. Current Snapshot: MvccSnapshot[applied={T|T < 7334742989340844032}]
I20260929 19:05:37.637446 1858 mvcc.cc:205] Tried to move back new op lower bound from 7334742989340844032 to 7334742989268086784. Current Snapshot: MvccSnapshot[applied={T|T < 7334742989340844032}]
I20260929 19:05:37.639989 575 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:37.640134 575 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:05:37.642573 575 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6abeb109cf0d4a4da444f9e29f8fd826: Sending DeleteTablet for 3 replicas of tablet b3ec4592a4c34d008d9d22ebc8dd7d0f
I20260929 19:05:37.642969 1480 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:37 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:49280
I20260929 19:05:37.642997 1754 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:37 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:48932
I20260929 19:05:37.643196 1869 tablet_replica.cc:330] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: stopping tablet replica
I20260929 19:05:37.643240 1868 tablet_replica.cc:330] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: stopping tablet replica
I20260929 19:05:37.643235 1618 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ec4592a4c34d008d9d22ebc8dd7d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:05:37 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:39506
I20260929 19:05:37.643298 1869 raft_consensus.cc:2229] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:37.643328 1868 raft_consensus.cc:2229] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:05:37.643399 1869 raft_consensus.cc:2258] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:37.643456 1868 raft_consensus.cc:2258] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:37.643870 1869 ts_tablet_manager.cc:1905] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:37.644006 1868 ts_tablet_manager.cc:1905] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:37.644906 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 543
I20260929 19:05:37.645869 1870 tablet_replica.cc:330] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: stopping tablet replica
I20260929 19:05:37.646025 1870 raft_consensus.cc:2229] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:05:37.646196 1870 raft_consensus.cc:2258] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:05:37.646561 1869 ts_tablet_manager.cc:1918] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:37.646636 1869 log.cc:1196] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/b3ec4592a4c34d008d9d22ebc8dd7d0f
I20260929 19:05:37.646919 1869 ts_tablet_manager.cc:1939] T b3ec4592a4c34d008d9d22ebc8dd7d0f P 1663f4cc14bf4d49b8f09597b73caf0b: Deleting consensus metadata
I20260929 19:05:37.647878 1870 ts_tablet_manager.cc:1905] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:05:37.649545 1870 ts_tablet_manager.cc:1918] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:37.649621 1870 log.cc:1196] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/b3ec4592a4c34d008d9d22ebc8dd7d0f
I20260929 19:05:37.649941 1870 ts_tablet_manager.cc:1939] T b3ec4592a4c34d008d9d22ebc8dd7d0f P d86b1258564348998c1cc67688a4111d: Deleting consensus metadata
W20260929 19:05:37.652268 1435 connection.cc:535] server connection from 127.0.0.1:49280 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 19:05:37.652540 1571 connection.cc:535] server connection from 127.0.0.1:39506 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:05:37.654291 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.644436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.26.78.252:44833)
0929 19:05:37.644479 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.644481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.644714 (+ 233us) server_negotiation.cc:407] Connection header received
0929 19:05:37.644822 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.644823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.644838 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.644869 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.646328 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.647154 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.648664 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.649014 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.649058 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.651004 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.651006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.651006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.651014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.652827 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.653255 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.653256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.653257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.653321 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.653551 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.653552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.653552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.653698 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.653774 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.654101 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.654194 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:05:37.654835 1868 ts_tablet_manager.cc:1918] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:05:37.654927 1868 log.cc:1196] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/b3ec4592a4c34d008d9d22ebc8dd7d0f
I20260929 19:05:37.655232 1868 ts_tablet_manager.cc:1939] T b3ec4592a4c34d008d9d22ebc8dd7d0f P a526a417a8e146319a0e1b6ea63c9004: Deleting consensus metadata
W20260929 19:05:37.655599 1707 connection.cc:441] server connection from 127.0.0.1:48932 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48932 (request call id 1) could send its response
I20260929 19:05:37.663404 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.654817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.26.78.253:36035)
0929 19:05:37.654857 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.654859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.655173 (+ 314us) server_negotiation.cc:407] Connection header received
0929 19:05:37.655267 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.655268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.655283 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.655327 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.656378 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.656861 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.657955 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.658276 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.658350 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.659855 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.659857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.659857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.659864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.661853 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.662212 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.662214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.662214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.662273 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.662809 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.662810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.662810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.662951 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.663056 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.663216 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.663306 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:05:37.664623 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:37.655180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.26.78.252:44833)
0929 19:05:37.655217 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:05:37.655219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:05:37.657998 (+ 2779us) server_negotiation.cc:407] Connection header received
0929 19:05:37.658052 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:05:37.658053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:05:37.658064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:05:37.658112 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:05:37.658820 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.659320 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.660071 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:05:37.660375 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:05:37.660442 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:05:37.661869 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:05:37.661870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:05:37.661870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:05:37.661877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:05:37.663530 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.663925 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.663926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.663927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.663986 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:05:37.664166 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:05:37.664167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:05:37.664168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:05:37.664291 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:05:37.664388 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 19:05:37.664448 (+ 60us) server_negotiation.cc:299] Negotiation successful
0929 19:05:37.664531 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:05:38.637277 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:38.636951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:33381)
0929 19:05:38.637123 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:38.637216 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
W20260929 19:05:38.637643 1820 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:44725 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:38.637933 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:38.637624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:33769)
0929 19:05:38.637772 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:38.637884 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":52,"threads_started":1}
W20260929 19:05:38.638193 1547 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:44725 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:38.638463 1880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:38.638197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:44159)
0929 19:05:38.638352 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:38.638416 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":49,"threads_started":1}
W20260929 19:05:38.638671 1684 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:44725 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:39.143795 1881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6abeb109cf0d4a4da444f9e29f8fd826)
I20260929 19:05:39.143929 1881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:39.144136 1881 leader_election.cc:288] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725)
I20260929 19:05:39.144623 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:39.144420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40070)
0929 19:05:39.144526 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:39.144585 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":63,"threads_started":1}
W20260929 19:05:39.144804 1161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:39.145223 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:39.145090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40084)
0929 19:05:39.145171 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:39.145201 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260929 19:05:39.145335 1161 leader_election.cc:334] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725): Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:39.145416 1161 leader_election.cc:302] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7842c1473cb44de6b63e14ddc22c71e6; no voters: 6abeb109cf0d4a4da444f9e29f8fd826
I20260929 19:05:39.145659 1881 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:05:39.638275 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:39.638048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:41739)
0929 19:05:39.638173 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:39.638219 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260929 19:05:39.638692 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:39.638495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:34887)
0929 19:05:39.638612 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:39.638646 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
I20260929 19:05:39.639197 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:39.638960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:55457)
0929 19:05:39.639091 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:39.639153 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260929 19:05:40.639122 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:40.638849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:54391)
0929 19:05:40.639006 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:40.639052 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260929 19:05:40.639500 1888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:40.639118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:33745)
0929 19:05:40.639410 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:40.639450 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":77,"threads_started":1}
I20260929 19:05:40.639889 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:40.639711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:55333)
0929 19:05:40.639826 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:40.639858 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260929 19:05:40.977313 1890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6abeb109cf0d4a4da444f9e29f8fd826)
I20260929 19:05:40.977421 1890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:40.977578 1890 leader_election.cc:288] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725)
I20260929 19:05:40.977981 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:40.977737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40340)
0929 19:05:40.977891 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:40.977943 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":47,"threads_started":1}
I20260929 19:05:40.978443 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:40.978340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40352)
0929 19:05:40.978388 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:40.978420 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 19:05:40.978530 1161 leader_election.cc:334] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725): Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:40.978596 1161 leader_election.cc:302] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7842c1473cb44de6b63e14ddc22c71e6; no voters: 6abeb109cf0d4a4da444f9e29f8fd826
I20260929 19:05:40.978686 1890 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:05:41.640159 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:41.639819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:36845)
0929 19:05:41.640044 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:41.640090 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":63,"threads_started":1}
I20260929 19:05:41.640412 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:41.639892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:42241)
0929 19:05:41.640330 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:41.640368 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":53,"threads_started":1}
I20260929 19:05:41.640502 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:41.640319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:53785)
0929 19:05:41.640427 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:41.640460 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":46,"threads_started":1}
I20260929 19:05:42.640944 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:42.640687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:55803)
0929 19:05:42.640834 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:42.640884 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260929 19:05:42.641129 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:42.640919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:60355)
0929 19:05:42.641043 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:42.641082 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":53,"threads_started":1}
I20260929 19:05:42.641263 1898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:42.640926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:37469)
0929 19:05:42.641178 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:42.641217 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":52,"threads_started":1}
I20260929 19:05:42.914641 1899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6abeb109cf0d4a4da444f9e29f8fd826)
I20260929 19:05:42.914736 1899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:42.914863 1899 leader_election.cc:288] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725)
I20260929 19:05:42.915243 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:42.915054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40538)
0929 19:05:42.915158 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:42.915204 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
I20260929 19:05:42.915653 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:42.915557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40542)
0929 19:05:42.915606 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:42.915634 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 19:05:42.915745 1161 leader_election.cc:334] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725): Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:42.915835 1161 leader_election.cc:302] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7842c1473cb44de6b63e14ddc22c71e6; no voters: 6abeb109cf0d4a4da444f9e29f8fd826
I20260929 19:05:42.915930 1899 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:05:43.641772 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:43.641480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:46887)
0929 19:05:43.641664 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:43.641710 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
I20260929 19:05:43.642169 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:43.641867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:37395)
0929 19:05:43.642061 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:43.642125 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":53,"threads_started":1}
I20260929 19:05:43.642215 1903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:43.641754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:42221)
0929 19:05:43.642124 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:43.642173 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":92,"threads_started":1}
I20260929 19:05:44.428196 1905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6abeb109cf0d4a4da444f9e29f8fd826)
I20260929 19:05:44.428321 1905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:44.428535 1905 leader_election.cc:288] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725)
I20260929 19:05:44.428866 1906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:44.428679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40682)
0929 19:05:44.428783 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:44.428823 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":63,"threads_started":1}
W20260929 19:05:44.428946 1161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260929 19:05:44.429282 1906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:44.429179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:40694)
0929 19:05:44.429235 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:44.429259 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 19:05:44.429371 1161 leader_election.cc:334] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725): Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:44.429414 1161 leader_election.cc:302] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7842c1473cb44de6b63e14ddc22c71e6; no voters: 6abeb109cf0d4a4da444f9e29f8fd826
I20260929 19:05:44.429484 1905 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:05:44.642655 1908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:44.642422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:59413)
0929 19:05:44.642552 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:44.642593 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260929 19:05:44.642928 1910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:44.642750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:39353)
0929 19:05:44.642849 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:44.642881 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":42,"threads_started":1}
I20260929 19:05:44.643237 1909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:44.642602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:47043)
0929 19:05:44.643151 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:44.643187 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":73,"threads_started":1}
I20260929 19:05:45.643900 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:45.643436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:35655)
0929 19:05:45.643759 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:45.643847 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":59,"threads_started":1}
I20260929 19:05:45.643900 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:45.643368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:51665)
0929 19:05:45.643820 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:45.643851 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":72,"threads_started":1}
I20260929 19:05:45.644259 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:45.643885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:43395)
0929 19:05:45.644164 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:45.644202 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":51,"threads_started":1}
I20260929 19:05:46.644732 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:46.644440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:40513)
0929 19:05:46.644615 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:46.644665 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":59,"threads_started":1}
I20260929 19:05:46.644807 1914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:46.644352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:55431)
0929 19:05:46.644721 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:46.644760 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":67,"threads_started":1}
I20260929 19:05:46.645347 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:46.645121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:57115)
0929 19:05:46.645270 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:46.645310 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260929 19:05:47.645694 1918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:47.645305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:59823)
0929 19:05:47.645591 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:47.645637 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":58,"threads_started":1}
I20260929 19:05:47.645689 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:47.645196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:47809)
0929 19:05:47.645579 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:47.645629 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":69,"threads_started":1}
I20260929 19:05:47.646128 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:47.645924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:35803)
0929 19:05:47.646050 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:47.646087 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":49,"threads_started":1}
I20260929 19:05:48.646721 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:48.646357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:52801)
0929 19:05:48.646599 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:48.646647 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":89,"threads_started":1}
I20260929 19:05:48.646981 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:48.646615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:33723)
0929 19:05:48.646879 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:48.646921 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":79,"threads_started":1}
I20260929 19:05:48.647119 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:48.646924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:43777)
0929 19:05:48.647030 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:48.647072 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
I20260929 19:05:49.422837 1923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6abeb109cf0d4a4da444f9e29f8fd826)
I20260929 19:05:49.422935 1923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:49.423131 1923 leader_election.cc:288] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725)
I20260929 19:05:49.423520 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:49.423316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:36772)
0929 19:05:49.423445 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:49.423480 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260929 19:05:49.424019 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:49.423891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:36780)
0929 19:05:49.423946 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:49.423977 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 19:05:49.424122 1161 leader_election.cc:334] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725): Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:49.424180 1161 leader_election.cc:302] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7842c1473cb44de6b63e14ddc22c71e6; no voters: 6abeb109cf0d4a4da444f9e29f8fd826
I20260929 19:05:49.424268 1923 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:05:49.647717 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:49.647471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:59717)
0929 19:05:49.647614 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:49.647658 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260929 19:05:49.647799 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:49.647541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:58795)
0929 19:05:49.647714 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:49.647750 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260929 19:05:49.647904 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:49.647541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:34139)
0929 19:05:49.647822 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:49.647860 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":62,"threads_started":1}
I20260929 19:05:50.648633 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:50.648400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:44351)
0929 19:05:50.648565 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:50.648595 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":54,"threads_started":1}
I20260929 19:05:50.648618 1930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:50.648383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:53705)
0929 19:05:50.648519 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:50.648562 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":46,"threads_started":1}
I20260929 19:05:50.648649 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:50.648336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:44893)
0929 19:05:50.648562 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:50.648608 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":67,"threads_started":1}
I20260929 19:05:51.649436 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:51.649118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:41067)
0929 19:05:51.649308 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:51.649382 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":65,"threads_started":1}
I20260929 19:05:51.649436 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:51.649158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:53519)
0929 19:05:51.649335 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:51.649378 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":59,"threads_started":1}
I20260929 19:05:51.649601 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:51.649399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:45345)
0929 19:05:51.649530 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:51.649560 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":45,"threads_started":1}
I20260929 19:05:52.650377 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:52.650084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:47927)
0929 19:05:52.650274 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:52.650325 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":60,"threads_started":1}
I20260929 19:05:52.650377 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:52.650073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:59095)
0929 19:05:52.650209 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:52.650334 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260929 19:05:52.650400 1935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:52.650073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:58775)
0929 19:05:52.650309 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:52.650351 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":62,"threads_started":1}
I20260929 19:05:53.651237 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:53.650817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:52201)
0929 19:05:53.651145 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:53.651183 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":65,"threads_started":1}
I20260929 19:05:53.651250 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:53.650976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:41551)
0929 19:05:53.651117 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:53.651159 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260929 19:05:53.651294 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:53.651033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:55645)
0929 19:05:53.651222 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:53.651252 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":48,"threads_started":1}
I20260929 19:05:54.652139 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:54.651686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:40903)
0929 19:05:54.651904 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:54.651987 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":65,"threads_started":1}
I20260929 19:05:54.652119 1942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:54.651875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:44273)
0929 19:05:54.652036 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:54.652072 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
I20260929 19:05:54.652323 1943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:54.651916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:51641)
0929 19:05:54.652158 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:54.652263 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":70,"threads_started":1}
I20260929 19:05:55.652994 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:55.652710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:49097)
0929 19:05:55.652875 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:55.652930 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":56,"threads_started":1}
I20260929 19:05:55.653038 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:55.652693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:47859)
0929 19:05:55.652919 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:55.652988 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":70,"threads_started":1}
I20260929 19:05:55.653276 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:55.653000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:56261)
0929 19:05:55.653187 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:55.653222 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":45,"threads_started":1}
I20260929 19:05:56.653961 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:56.653605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:56207)
0929 19:05:56.653785 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:56.653875 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":128,"threads_started":1}
I20260929 19:05:56.653960 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:56.653546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:47019)
0929 19:05:56.653807 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:56.653892 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":69,"threads_started":1}
I20260929 19:05:56.654210 1949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:56.653890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:33549)
0929 19:05:56.654023 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:56.654162 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":45,"threads_started":1}
I20260929 19:05:57.654881 1952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:57.654657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:50953)
0929 19:05:57.654783 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:57.654821 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":46,"threads_started":1}
I20260929 19:05:57.654870 1951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:57.654452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:50919)
0929 19:05:57.654767 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:57.654812 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":58,"threads_started":1}
I20260929 19:05:57.654912 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:57.654390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:58359)
0929 19:05:57.654837 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:57.654868 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":81,"threads_started":1}
I20260929 19:05:57.794036 1953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6abeb109cf0d4a4da444f9e29f8fd826)
I20260929 19:05:57.794137 1953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6abeb109cf0d4a4da444f9e29f8fd826" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44725 } } peers { permanent_uuid: "7842c1473cb44de6b63e14ddc22c71e6" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36035 } attrs { promote: false } } peers { permanent_uuid: "8d8f747ed6c9437aac8d74c8cf770758" member_type: NON_VOTER last_known_addr { host: "127.26.78.252" port: 44833 } attrs { promote: true } }
I20260929 19:05:57.794341 1953 leader_election.cc:288] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725)
I20260929 19:05:57.794706 1954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:57.794498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37218)
0929 19:05:57.794620 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:57.794667 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
W20260929 19:05:57.794795 1161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260929 19:05:57.795151 1954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:57.795052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.0.0.1:37224)
0929 19:05:57.795102 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:57.795127 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 19:05:57.795235 1161 leader_election.cc:334] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6abeb109cf0d4a4da444f9e29f8fd826 (127.26.78.254:44725): Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
I20260929 19:05:57.795279 1161 leader_election.cc:302] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7842c1473cb44de6b63e14ddc22c71e6; no voters: 6abeb109cf0d4a4da444f9e29f8fd826
I20260929 19:05:57.795348 1953 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7842c1473cb44de6b63e14ddc22c71e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:05:58.655685 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:58.655346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:55735)
0929 19:05:58.655500 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:58.655559 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260929 19:05:58.655714 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:58.655463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:60751)
0929 19:05:58.655627 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:58.655665 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260929 19:05:58.655725 1957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:58.655420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:60141)
0929 19:05:58.655566 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:58.655606 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":45,"threads_started":1}
I20260929 19:05:59.656459 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:59.656074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:56753)
0929 19:05:59.656360 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:59.656406 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":66,"threads_started":1}
I20260929 19:05:59.656454 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:59.656166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:58441)
0929 19:05:59.656366 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:59.656411 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
I20260929 19:05:59.656642 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:05:59.656263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:34311)
0929 19:05:59.656507 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 19:05:59.656601 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":63,"threads_started":1}
I20260929 19:06:00.657281 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:00.656965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:46735)
0929 19:06:00.657177 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:00.657223 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":67,"threads_started":1}
I20260929 19:06:00.657325 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:00.657111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:48985)
0929 19:06:00.657241 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:00.657270 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260929 19:06:00.657485 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:00.657273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:34199)
0929 19:06:00.657394 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:00.657433 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":47,"threads_started":1}
I20260929 19:06:01.658072 1966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:01.657844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:49263)
0929 19:06:01.657973 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:01.658014 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":55,"threads_started":1}
I20260929 19:06:01.658176 1967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:01.657978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:39535)
0929 19:06:01.658107 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:01.658137 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260929 19:06:01.658183 1965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:01.657772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:55633)
0929 19:06:01.658092 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:01.658133 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":82,"threads_started":1}
I20260929 19:06:02.658774 1969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:02.658539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:44281)
0929 19:06:02.658678 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:02.658716 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260929 19:06:02.658777 1968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:02.658539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:58579)
0929 19:06:02.658680 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:02.658723 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260929 19:06:02.658957 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:02.658676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:59135)
0929 19:06:02.658795 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:02.658845 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":54,"threads_started":1}
I20260929 19:06:03.659435 1971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:03.659196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:42911)
0929 19:06:03.659327 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:03.659373 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260929 19:06:03.659582 1972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:03.659361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:59357)
0929 19:06:03.659490 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:03.659530 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":56,"threads_started":1}
I20260929 19:06:03.659659 1973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:03.659446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:53521)
0929 19:06:03.659582 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:03.659615 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":48,"threads_started":1}
I20260929 19:06:04.660100 1974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:04.659840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:41227)
0929 19:06:04.659987 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:04.660039 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260929 19:06:04.660101 1975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:04.659911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:54827)
0929 19:06:04.660013 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:04.660048 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":45,"threads_started":1}
I20260929 19:06:04.660395 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:04.660189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:34293)
0929 19:06:04.660332 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:04.660365 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":46,"threads_started":1}
I20260929 19:06:05.660804 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:05.660512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:35243)
0929 19:06:05.660701 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:05.660747 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
I20260929 19:06:05.660807 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:05.660517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:33325)
0929 19:06:05.660700 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:05.660747 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260929 19:06:05.660899 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:05.660735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:56435)
0929 19:06:05.660830 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:05.660857 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":44,"threads_started":1}
I20260929 19:06:06.661532 1982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:06.661322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:39385)
0929 19:06:06.661461 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:06.661493 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260929 19:06:06.661532 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:06.661266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:41127)
0929 19:06:06.661431 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:06.661478 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260929 19:06:06.661685 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:06.661311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:49431)
0929 19:06:06.661592 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:06.661634 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":73,"threads_started":1}
I20260929 19:06:07.662300 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:07.662066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.194:44925)
0929 19:06:07.662192 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:07.662238 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260929 19:06:07.662407 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:07.662223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.193:58907)
0929 19:06:07.662339 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:07.662369 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
I20260929 19:06:07.662456 1984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:07.662066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44725 (local address 127.26.78.195:45503)
0929 19:06:07.662301 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:07.662351 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":69,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.26.78.254:44725,127.26.78.253:36035,127.26.78.252:44833, num_attempts: 1)
I20260929 19:06:07.868021 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 1321
I20260929 19:06:07.873169 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 1555
I20260929 19:06:07.877717 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 1691
I20260929 19:06:07.883086 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 1040
I20260929 19:06:07.887578 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 1228
2026-09-29T19:06:07Z chronyd exiting
I20260929 19:06:07.903538 26939 test_util.cc:175] -----------------------------------------------
I20260929 19:06:07.903610 26939 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790708710206825-26939-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33090 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:06:07 dist-test-slave-xz51 krb5kdc[1988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:06:07 dist-test-slave-xz51 krb5kdc[1988](info): set up 2 sockets
Sep 29 19:06:07 dist-test-slave-xz51 krb5kdc[1988](info): commencing operation
krb5kdc: starting...
W20260929 19:06:08.926523 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:06:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:06:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab.
Entry for principal oryx/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab.
I20260929 19:06:09.053371 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:37641
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab
--principal=oryx/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:37641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:09.138171 2001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:09.138384 2001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:09.138406 2001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:09.139576 2001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:06:09.139619 2001 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 19:06:09.139632 2001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:09.139645 2001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:09.139700 2001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:06:09.139717 2001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:06:09.141247 2001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:37641
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.254.keytab
--principal=oryx/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:37641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:09.141502 2001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:09.141742 2001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:09.146759 2007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:09.146723 2009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:09.146726 2006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:09.147102 2001 server_base.cc:1034] running on GCE node
I20260929 19:06:09.147347 2001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:09.147598 2001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:09.148744 2001 hybrid_clock.cc:648] HybridClock initialized: now 1790708769148723 us; error 42 us; skew 500 ppm
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:09.151522 2001 init.cc:377] Logged in from keytab as oryx/127.26.78.254@KRBTEST.COM (short username oryx)
I20260929 19:06:09.152114 2001 webserver.cc:460] Webserver started at http://127.26.78.254:40093/ using document root <none> and password file <none>
I20260929 19:06:09.152355 2001 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:09.152400 2001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:09.152556 2001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:09.153656 2001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "c931ed6eac8b4b19a1be17697d03e5ee"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "4b6c0ce70c81b84038a055319f5a7cf5"
server_key_iv: "a745639fb66f870b659c07ed4530ca93"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.154065 2001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "c931ed6eac8b4b19a1be17697d03e5ee"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "4b6c0ce70c81b84038a055319f5a7cf5"
server_key_iv: "a745639fb66f870b659c07ed4530ca93"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.155458 2001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:09.156330 2016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.156544 2001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:09.156688 2001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "c931ed6eac8b4b19a1be17697d03e5ee"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "4b6c0ce70c81b84038a055319f5a7cf5"
server_key_iv: "a745639fb66f870b659c07ed4530ca93"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.156769 2001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:09.173152 2001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:09.175894 2001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:09.176069 2001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:09.181392 2001 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:37641
I20260929 19:06:09.181435 2078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:37641 every 8 connection(s)
I20260929 19:06:09.181783 2001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:06:09.182551 2079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:09.184877 2079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Bootstrap starting.
I20260929 19:06:09.185504 2079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:09.185801 2079 log.cc:824] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:09.186561 2079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: No bootstrap required, opened a new log
I20260929 19:06:09.188470 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 2001
I20260929 19:06:09.188591 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:06:09.188793 26939 external_mini_cluster.cc:1428] Setting key 614626cd26ab926a128a7f1bb57056df
I20260929 19:06:09.188855 2079 raft_consensus.cc:348] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37641 } }
I20260929 19:06:09.189059 2079 raft_consensus.cc:374] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:09.189105 2079 raft_consensus.cc:729] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c931ed6eac8b4b19a1be17697d03e5ee, State: Initialized, Role: FOLLOWER
I20260929 19:06:09.189280 2079 consensus_queue.cc:260] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37641 } }
I20260929 19:06:09.189414 2079 raft_consensus.cc:388] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:06:09.189525 2079 raft_consensus.cc:482] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:06:09.189600 2079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:09.190207 2079 raft_consensus.cc:504] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37641 } }
I20260929 19:06:09.190333 2079 leader_election.cc:302] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c931ed6eac8b4b19a1be17697d03e5ee; no voters:
I20260929 19:06:09.190508 2079 leader_election.cc:288] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:06:09.190550 2084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:09.190655 2084 raft_consensus.cc:686] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [term 1 LEADER]: Becoming Leader. State: Replica: c931ed6eac8b4b19a1be17697d03e5ee, State: Running, Role: LEADER
I20260929 19:06:09.190779 2079 sys_catalog.cc:564] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:06:09.190793 2084 consensus_queue.cc:237] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37641 } }
I20260929 19:06:09.191201 2084 sys_catalog.cc:455] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader c931ed6eac8b4b19a1be17697d03e5ee. Latest consensus state: current_term: 1 leader_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37641 } } }
I20260929 19:06:09.191210 2086 sys_catalog.cc:455] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c931ed6eac8b4b19a1be17697d03e5ee" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37641 } } }
I20260929 19:06:09.191413 2086 sys_catalog.cc:458] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:09.191624 2084 sys_catalog.cc:458] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:09.191988 2093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:06:09.192505 2093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:06:09.194315 2093 catalog_manager.cc:1347] Generated new cluster ID: e9bd0a0651274a6da1544b736968f848
I20260929 19:06:09.194380 2093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:06:09.201447 2093 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:06:09.202078 2093 catalog_manager.cc:1504] Loading token signing keys...
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:06:09.207968 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.189343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.26.78.254:37641)
0929 19:06:09.189522 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.189532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.189571 (+ 39us) server_negotiation.cc:407] Connection header received
0929 19:06:09.189719 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.189732 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.189775 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.189937 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.190702 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.192155 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.193306 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.193982 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.194095 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.204052 (+ 9957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.204056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.204057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.204069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.206172 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.206607 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.206611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.206611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.206727 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.206997 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.206998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.206998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.207280 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.207414 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.207562 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.207727 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260929 19:06:09.209947 2093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Generated new TSK 0
I20260929 19:06:09.210129 2093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab.
Entry for principal oryx/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab.
I20260929 19:06:09.245316 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:37641
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab
--principal=oryx/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:09.332783 2105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:09.333002 2105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:09.333027 2105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:09.334264 2105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:09.334308 2105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:09.334329 2105 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:06:09.334347 2105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:06:09.335912 2105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab
--principal=oryx/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:37641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:09.336155 2105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:09.336413 2105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:09.341692 2110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:09.341692 2111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:09.341723 2105 server_base.cc:1034] running on GCE node
W20260929 19:06:09.341713 2113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:09.342232 2105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:09.342496 2105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:09.343665 2105 hybrid_clock.cc:648] HybridClock initialized: now 1790708769343638 us; error 60 us; skew 500 ppm
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:09.346411 2105 init.cc:377] Logged in from keytab as oryx/127.26.78.193@KRBTEST.COM (short username oryx)
I20260929 19:06:09.346866 2105 webserver.cc:460] Webserver started at http://127.26.78.193:43555/ using document root <none> and password file <none>
I20260929 19:06:09.347110 2105 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:09.347157 2105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:09.347297 2105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:09.348348 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "5f41db56342a4bb9999508da3eeb2e9c"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "01fcb975c5183bbd1b0f80e8d8f6eeeb"
server_key_iv: "94ac1c26c4bdafe15c2b2eef61c0aa2d"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.348716 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "5f41db56342a4bb9999508da3eeb2e9c"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "01fcb975c5183bbd1b0f80e8d8f6eeeb"
server_key_iv: "94ac1c26c4bdafe15c2b2eef61c0aa2d"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.350323 2105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:06:09.351078 2120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.351243 2105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:09.351336 2105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "5f41db56342a4bb9999508da3eeb2e9c"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "01fcb975c5183bbd1b0f80e8d8f6eeeb"
server_key_iv: "94ac1c26c4bdafe15c2b2eef61c0aa2d"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.351428 2105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:09.375247 2105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:09.377300 2105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:09.377482 2105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:09.378044 2105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:09.378109 2105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.378154 2105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:09.378185 2105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.385422 2105 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:37615
I20260929 19:06:09.385488 2236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:37615 every 8 connection(s)
I20260929 19:06:09.385878 2105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:06:09.389173 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.378611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:58197 (local address 127.26.78.254:37641)
0929 19:06:09.378685 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.378687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.378924 (+ 237us) server_negotiation.cc:407] Connection header received
0929 19:06:09.379728 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.379730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.379745 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.379819 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.380797 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.381702 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.383441 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.383637 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.383696 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.386528 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.386529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.386529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.386535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.388040 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.388396 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.388397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.388397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.388447 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.388658 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.388658 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.388659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.388756 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.388804 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.389043 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.389110 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:06:09.389360 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.378591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:37641 (local address 127.26.78.193:58197)
0929 19:06:09.378770 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.378820 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.379600 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.379907 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.379912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.380378 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:09.380687 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.380695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.381810 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.381812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.383353 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.383356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.383728 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.383729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.383884 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.384547 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:09.384558 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:09.386443 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:09.388111 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.388115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.388116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.388332 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.388504 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.388505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.388505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.388599 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.388809 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:09.388813 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:09.388932 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.389015 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.389135 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260929 19:06:09.389874 2237 heartbeater.cc:347] Connected to a master server at 127.26.78.254:37641
I20260929 19:06:09.389985 2237 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:09.390166 2237 heartbeater.cc:511] Master 127.26.78.254:37641 requested a full tablet report, sending...
I20260929 19:06:09.390763 2032 ts_manager.cc:194] Registered new tserver with Master: 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615)
I20260929 19:06:09.391350 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 2105
I20260929 19:06:09.391453 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:06:09.391666 26939 external_mini_cluster.cc:1428] Setting key 2bd6935fef3211973125aac2f2dcc4c1
I20260929 19:06:09.391705 2032 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:58197
I20260929 19:06:09.397315 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.390939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:37641 (local address 127.26.78.193:42685)
0929 19:06:09.391131 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.391143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.391216 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.391365 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.391366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.391796 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:09.391882 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.391884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.392798 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.392799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.393295 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.393296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.393782 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.393783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.393879 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.394325 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:09.394331 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:09.394619 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:09.396377 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.396378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.396378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.396538 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.396739 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.396740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.396740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.396791 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.397077 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:09.397078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:09.397112 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.397170 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.397243 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":96,"threads_started":1}
I20260929 19:06:09.397439 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.391044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:42685 (local address 127.26.78.254:37641)
0929 19:06:09.391083 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.391085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.391194 (+ 109us) server_negotiation.cc:407] Connection header received
0929 19:06:09.391277 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.391278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.391287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.391313 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.391977 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.392699 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.393389 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.393698 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.393769 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.394710 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.394711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.394712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.394719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.396283 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.396613 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.396614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.396614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.396673 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.396857 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.396858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.396858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.397005 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.397075 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.397213 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.397294 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab.
Entry for principal oryx/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab.
I20260929 19:06:09.417866 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:37641
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab
--principal=oryx/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:09.499004 2246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:09.499277 2246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:09.499321 2246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:09.500535 2246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:09.500604 2246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:09.500639 2246 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:06:09.500689 2246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:06:09.502255 2246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab
--principal=oryx/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:37641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:09.502524 2246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:09.502765 2246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:09.507822 2252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:09.507933 2246 server_base.cc:1034] running on GCE node
W20260929 19:06:09.507881 2254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:09.507822 2251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:09.508234 2246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:09.508494 2246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:09.509667 2246 hybrid_clock.cc:648] HybridClock initialized: now 1790708769509581 us; error 127 us; skew 500 ppm
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:09.512382 2246 init.cc:377] Logged in from keytab as oryx/127.26.78.194@KRBTEST.COM (short username oryx)
I20260929 19:06:09.512816 2246 webserver.cc:460] Webserver started at http://127.26.78.194:38477/ using document root <none> and password file <none>
I20260929 19:06:09.513079 2246 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:09.513145 2246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:09.513288 2246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:09.514389 2246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "9aac166a9da8370e9e9020290080db13"
server_key_iv: "9a924eccf9cd967a939142a2b32ca5c6"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.514748 2246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "9aac166a9da8370e9e9020290080db13"
server_key_iv: "9a924eccf9cd967a939142a2b32ca5c6"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.516074 2246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:06:09.516799 2261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.516991 2246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:09.517091 2246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "9aac166a9da8370e9e9020290080db13"
server_key_iv: "9a924eccf9cd967a939142a2b32ca5c6"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.517160 2246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:09.532898 2246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:09.535394 2246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:09.535571 2246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:09.536109 2246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:09.536177 2246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.536216 2246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:09.536243 2246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.543116 2246 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:41029
I20260929 19:06:09.543201 2376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:41029 every 8 connection(s)
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:06:09.543526 2246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:06:09.546715 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.536628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:52509 (local address 127.26.78.254:37641)
0929 19:06:09.536676 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.536678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.536930 (+ 252us) server_negotiation.cc:407] Connection header received
0929 19:06:09.537517 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.537519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.537534 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.537610 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.538569 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.539504 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.541080 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.541345 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.541425 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.543996 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.543998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.543998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.544007 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.545318 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.545777 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.545778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.545778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.545901 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.546163 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.546164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.546164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.546283 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.546340 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.546579 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.546648 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:06:09.546895 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.536630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:37641 (local address 127.26.78.194:52509)
0929 19:06:09.536802 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.536839 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.537367 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.537754 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.537759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.538149 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:09.538450 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.538458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.539607 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.539608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.541002 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.541006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.541432 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.541433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.541590 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.542250 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:09.542260 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:09.543884 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:09.545413 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.545419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.545421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.545678 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.545992 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.545993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.545994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.546099 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.546347 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:09.546352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:09.546465 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.546550 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.546670 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260929 19:06:09.547379 2377 heartbeater.cc:347] Connected to a master server at 127.26.78.254:37641
I20260929 19:06:09.547465 2377 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:09.547654 2377 heartbeater.cc:511] Master 127.26.78.254:37641 requested a full tablet report, sending...
I20260929 19:06:09.548026 2033 ts_manager.cc:194] Registered new tserver with Master: 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029)
I20260929 19:06:09.548525 2033 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:52509
I20260929 19:06:09.552874 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 2246
I20260929 19:06:09.552989 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:06:09.553231 26939 external_mini_cluster.cc:1428] Setting key b0863c40b7821d24b4ba0a032aaaf139
I20260929 19:06:09.553969 2383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.548413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:37641 (local address 127.26.78.194:59747)
0929 19:06:09.548534 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.548545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.548606 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.548756 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.548757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.548965 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:09.549036 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.549039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.549659 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.549660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.550224 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.550226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.550541 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.550542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.550587 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.550955 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:09.550959 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:09.551216 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:09.552736 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.552737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.552738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.552961 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.553205 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.553206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.553206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.553263 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.553636 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:09.553637 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:09.553675 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.553731 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.553864 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
I20260929 19:06:09.553987 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.548481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:59747 (local address 127.26.78.254:37641)
0929 19:06:09.548523 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.548525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.548590 (+ 65us) server_negotiation.cc:407] Connection header received
0929 19:06:09.548684 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.548685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.548694 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.548748 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.549117 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.549572 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.550290 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.550476 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.550529 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.551287 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.551288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.551289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.551294 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.552665 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.553064 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.553065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.553066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.553127 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.553357 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.553358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.553358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.553560 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.553633 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.553773 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.553895 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab.
Entry for principal oryx/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab.
I20260929 19:06:09.580657 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:37641
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab
--principal=oryx/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:09.662773 2386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:09.663018 2386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:09.663058 2386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:09.664211 2386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:09.664284 2386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:09.664317 2386 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:06:09.664356 2386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:06:09.665952 2386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab
--principal=oryx/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.78.254:37641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:09.666234 2386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:09.666471 2386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:09.671459 2392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:09.671474 2391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:09.671473 2394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:09.671797 2386 server_base.cc:1034] running on GCE node
I20260929 19:06:09.672045 2386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:09.672293 2386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:09.673448 2386 hybrid_clock.cc:648] HybridClock initialized: now 1790708769673433 us; error 28 us; skew 500 ppm
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:09.676247 2386 init.cc:377] Logged in from keytab as oryx/127.26.78.195@KRBTEST.COM (short username oryx)
I20260929 19:06:09.676738 2386 webserver.cc:460] Webserver started at http://127.26.78.195:46835/ using document root <none> and password file <none>
I20260929 19:06:09.676988 2386 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:09.677058 2386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:09.677210 2386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:09.678345 2386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "69dab811c9e2400556c9be947f7d9fe6"
server_key_iv: "ac4f5f7bab6b7115127e0f5357b91568"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.678695 2386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "69dab811c9e2400556c9be947f7d9fe6"
server_key_iv: "ac4f5f7bab6b7115127e0f5357b91568"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.679993 2386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:06:09.680773 2401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.680954 2386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:09.681057 2386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7"
format_stamp: "Formatted at 2026-09-29 19:06:09 on dist-test-slave-xz51"
server_key: "69dab811c9e2400556c9be947f7d9fe6"
server_key_iv: "ac4f5f7bab6b7115127e0f5357b91568"
server_key_version: "encryptionkey@0"
I20260929 19:06:09.681135 2386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:09.698206 2386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:09.700605 2386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:09.700788 2386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:09.701395 2386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:09.701455 2386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:09.701498 2386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:09.701519 2386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:06:09.709038 2386 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:42443
I20260929 19:06:09.709123 2516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:42443 every 8 connection(s)
I20260929 19:06:09.709468 2386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:06:09.711827 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.701805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:57525 (local address 127.26.78.254:37641)
0929 19:06:09.701914 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.701917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.702131 (+ 214us) server_negotiation.cc:407] Connection header received
0929 19:06:09.702882 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.702883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.702897 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.702962 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.704021 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.704660 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.706369 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.706569 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.706651 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.709116 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.709118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.709118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.709126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.710711 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.711073 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.711074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.711074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.711123 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.711326 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.711327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.711327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.711440 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.711489 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.711705 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.711765 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 19:06:09.712001 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.701762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:37641 (local address 127.26.78.195:57525)
0929 19:06:09.701967 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.702025 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.702757 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.703048 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.703052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.703482 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:09.703853 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.703860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.704732 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.704734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.706269 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.706272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.706646 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.706647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.706811 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.707332 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:09.707344 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:09.709007 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:09.710782 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.710788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.710790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.711006 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.711179 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.711179 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.711180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.711273 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.711493 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:09.711498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:09.711610 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.711692 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.711808 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":123,"threads_started":1}
I20260929 19:06:09.712512 2517 heartbeater.cc:347] Connected to a master server at 127.26.78.254:37641
I20260929 19:06:09.712590 2517 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:09.712769 2517 heartbeater.cc:511] Master 127.26.78.254:37641 requested a full tablet report, sending...
I20260929 19:06:09.713137 2032 ts_manager.cc:194] Registered new tserver with Master: 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443)
I20260929 19:06:09.713598 2032 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:57525
I20260929 19:06:09.715969 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 2386
I20260929 19:06:09.716070 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:06:09.716276 26939 external_mini_cluster.cc:1428] Setting key 43f0923be3c86a2f7ce394be5557b5cc
I20260929 19:06:09.717015 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:06:09.719205 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.713601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:35637 (local address 127.26.78.254:37641)
0929 19:06:09.713641 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.713643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.713701 (+ 58us) server_negotiation.cc:407] Connection header received
0929 19:06:09.713784 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.713785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.713796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.713876 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.714209 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.714670 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.715384 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.715580 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.715663 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.716581 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.716582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.716583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.716590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.718078 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.718440 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.718441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.718442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.718501 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.718698 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.718699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.718699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.718809 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.718877 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.719043 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.719109 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:06:09.719214 2523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.713497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:37641 (local address 127.26.78.195:35637)
0929 19:06:09.713643 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.713654 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.713715 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.713882 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.713883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.714062 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:09.714123 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.714124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.714784 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.714785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.715285 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.715287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.715665 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.715666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.715732 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.716185 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:09.716189 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:09.716500 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:09.718175 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.718176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.718176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.718352 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.718567 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:09.718568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:09.718568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:09.718627 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:09.718880 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:09.718882 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:09.718928 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.719007 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.719112 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 19:06:09.738611 2083 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:09.733396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59838 (local address 127.26.78.254:37641)
0929 19:06:09.733459 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.733461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.733478 (+ 17us) server_negotiation.cc:407] Connection header received
0929 19:06:09.733505 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.733506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.733514 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.733564 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.734157 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.734642 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.735345 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.735520 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.735568 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.738524 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59838: BlockingRecv error: recv got EOF from 127.0.0.1:59838 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 29 19:06:09 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:06:09.747749 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.739439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.26.78.254:37641)
0929 19:06:09.739513 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.739514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.739530 (+ 16us) server_negotiation.cc:407] Connection header received
0929 19:06:09.739595 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.739596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.739605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.739651 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:09.740066 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.740529 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.741177 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.741391 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.741455 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.744388 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:09.744389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:09.744390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:09.744397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:09.746305 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.746762 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.746763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.746764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.746829 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:09.747057 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:09.747058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:09.747058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:09.747269 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:09.747405 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.747538 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.747641 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:06:09.749697 2032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:06:09.751549 2032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:06:09.758685 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.754218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.26.78.194:41029)
0929 19:06:09.754593 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.754597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.754607 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:06:09.754648 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.754650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.754687 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.755191 (+ 504us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:09.755348 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.756372 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.757456 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.758230 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.758336 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.758418 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.758537 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.758574 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":79,"threads_started":1}
I20260929 19:06:09.758702 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.754084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:37615 (local address 127.0.0.1:41536)
0929 19:06:09.754409 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.754420 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.754472 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.755032 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.755033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.755035 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:09.755076 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.755077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.756029 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.756030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.757370 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.757373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.758440 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.758441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.758518 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.758521 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.758565 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.758617 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":12,"thread_start_us":87,"threads_started":1}
I20260929 19:06:09.758822 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.754384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33142 (local address 127.26.78.195:42443)
0929 19:06:09.754559 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.754564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.754601 (+ 37us) server_negotiation.cc:407] Connection header received
0929 19:06:09.754658 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.754660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.754674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.754762 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:09.755058 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.756175 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.757475 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.758248 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.758322 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.758402 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.758661 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.758700 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260929 19:06:09.758967 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.754284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:42443 (local address 127.0.0.1:33142)
0929 19:06:09.754546 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.754555 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.754606 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.754888 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.754892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.754895 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:09.754980 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.754982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.756262 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.756263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.757371 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.757373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.758441 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.758442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.758509 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.758514 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.758565 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.758617 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":17,"thread_start_us":78,"threads_started":1}
I20260929 19:06:09.759059 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.754084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41029 (local address 127.0.0.1:38046)
0929 19:06:09.754234 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.754289 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.754362 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.755204 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.755205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.755207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:09.755266 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.755268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.756458 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.756459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.757366 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.757368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.758341 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.758342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.758431 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.758446 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.758522 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.758580 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":2,"thread_start_us":49,"threads_started":1}
I20260929 19:06:09.759295 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.754399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.26.78.193:37615)
0929 19:06:09.754831 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.754835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.754844 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:09.754881 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.754883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.754896 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.754970 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:09.755152 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.755934 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.757475 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.758247 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.758322 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.758402 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.759136 (+ 734us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.759184 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":71,"threads_started":1}
I20260929 19:06:09.759732 2311 tablet_service.cc:1467] Processing CreateTablet for tablet 026645269d7f4a5e909a4519036f61a5 (DEFAULT_TABLE table=test-table [id=131da00829414245a4e7b86fb6ec9fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:09.760103 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026645269d7f4a5e909a4519036f61a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:09.763659 2545 tablet_bootstrap.cc:492] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Bootstrap starting.
I20260929 19:06:09.764128 2449 tablet_service.cc:1467] Processing CreateTablet for tablet 026645269d7f4a5e909a4519036f61a5 (DEFAULT_TABLE table=test-table [id=131da00829414245a4e7b86fb6ec9fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:09.764454 2545 tablet_bootstrap.cc:654] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:09.764493 2449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026645269d7f4a5e909a4519036f61a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:09.764803 2545 log.cc:824] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:09.765590 2545 tablet_bootstrap.cc:492] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: No bootstrap required, opened a new log
I20260929 19:06:09.765647 2545 ts_tablet_manager.cc:1397] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:06:09.766544 2153 tablet_service.cc:1467] Processing CreateTablet for tablet 026645269d7f4a5e909a4519036f61a5 (DEFAULT_TABLE table=test-table [id=131da00829414245a4e7b86fb6ec9fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:09.766892 2153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026645269d7f4a5e909a4519036f61a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:09.767765 2547 tablet_bootstrap.cc:492] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Bootstrap starting.
I20260929 19:06:09.767684 2545 raft_consensus.cc:348] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.767869 2545 raft_consensus.cc:374] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:09.767911 2545 raft_consensus.cc:729] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b9e3bef2dd4e848b166cf89fd9e2f3, State: Initialized, Role: FOLLOWER
I20260929 19:06:09.768043 2545 consensus_queue.cc:260] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.768335 2545 ts_tablet_manager.cc:1428] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:06:09.768371 2377 heartbeater.cc:503] Master 127.26.78.254:37641 was elected leader, sending a full tablet report...
I20260929 19:06:09.768685 2547 tablet_bootstrap.cc:654] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:09.769016 2547 log.cc:824] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:09.769783 2547 tablet_bootstrap.cc:492] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: No bootstrap required, opened a new log
I20260929 19:06:09.769893 2547 ts_tablet_manager.cc:1397] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:06:09.770709 2550 tablet_bootstrap.cc:492] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Bootstrap starting.
I20260929 19:06:09.771515 2550 tablet_bootstrap.cc:654] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:09.771631 2547 raft_consensus.cc:348] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.771796 2547 raft_consensus.cc:374] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:09.771842 2547 raft_consensus.cc:729] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 719cfe1acf5e42f8b7d0effaee3bfbb7, State: Initialized, Role: FOLLOWER
I20260929 19:06:09.771853 2550 log.cc:824] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:09.771958 2547 consensus_queue.cc:260] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.772254 2547 ts_tablet_manager.cc:1428] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:06:09.772261 2517 heartbeater.cc:503] Master 127.26.78.254:37641 was elected leader, sending a full tablet report...
I20260929 19:06:09.772572 2550 tablet_bootstrap.cc:492] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: No bootstrap required, opened a new log
I20260929 19:06:09.772799 2550 ts_tablet_manager.cc:1397] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:06:09.774353 2550 raft_consensus.cc:348] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.774500 2550 raft_consensus.cc:374] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:09.774547 2550 raft_consensus.cc:729] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f41db56342a4bb9999508da3eeb2e9c, State: Initialized, Role: FOLLOWER
I20260929 19:06:09.774664 2550 consensus_queue.cc:260] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.774941 2550 ts_tablet_manager.cc:1428] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:06:09.775012 2237 heartbeater.cc:503] Master 127.26.78.254:37641 was elected leader, sending a full tablet report...
W20260929 19:06:09.794127 2378 tablet.cc:2369] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:06:09.797320 2548 raft_consensus.cc:482] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:06:09.797423 2548 raft_consensus.cc:504] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.797721 2548 leader_election.cc:288] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443)
I20260929 19:06:09.802062 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.798309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:46905 (local address 127.26.78.193:37615)
0929 19:06:09.798413 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.798415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.798424 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:09.798451 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.798453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.798463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.798528 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:09.798728 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.799489 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.801027 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.801638 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.801722 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.801730 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.801948 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.801989 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 19:06:09.802062 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.798312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:41177 (local address 127.26.78.195:42443)
0929 19:06:09.798386 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.798389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.798402 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:06:09.798467 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.798468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.798477 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.798530 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:09.798728 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.799489 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.801028 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.801648 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.801721 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.801730 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.801948 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.801989 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:06:09.802105 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.798173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:42443 (local address 127.26.78.194:41177)
0929 19:06:09.798358 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.798368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.798420 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.798555 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.798556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.798558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:09.798646 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.798648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.799574 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.799575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.800917 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.800920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.801743 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.801744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.801872 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.801875 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.801930 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.801977 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":90,"thread_start_us":125,"threads_started":1}
I20260929 19:06:09.802111 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.798060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:37615 (local address 127.26.78.194:46905)
0929 19:06:09.798117 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:09.798129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:09.798194 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:09.798555 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:09.798556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:09.798558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:09.798646 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.798648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.799574 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.799575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.800917 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:09.800920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.801742 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.801744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:09.801863 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.801868 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:06:09.801930 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 19:06:09.801983 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":107}
I20260929 19:06:09.802479 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026645269d7f4a5e909a4519036f61a5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" is_pre_election: true
I20260929 19:06:09.802649 2471 raft_consensus.cc:2454] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 0.
I20260929 19:06:09.802673 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026645269d7f4a5e909a4519036f61a5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c" is_pre_election: true
I20260929 19:06:09.802822 2191 raft_consensus.cc:2454] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 0.
I20260929 19:06:09.802923 2263 leader_election.cc:302] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b9e3bef2dd4e848b166cf89fd9e2f3, 719cfe1acf5e42f8b7d0effaee3bfbb7; no voters:
I20260929 19:06:09.803095 2548 raft_consensus.cc:2781] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:06:09.803179 2548 raft_consensus.cc:482] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:06:09.803217 2548 raft_consensus.cc:3037] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:09.804258 2548 raft_consensus.cc:504] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.804397 2548 leader_election.cc:288] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 election: Requested vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443)
I20260929 19:06:09.804607 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026645269d7f4a5e909a4519036f61a5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c"
I20260929 19:06:09.804611 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026645269d7f4a5e909a4519036f61a5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7"
I20260929 19:06:09.804685 2191 raft_consensus.cc:3037] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:09.804695 2471 raft_consensus.cc:3037] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:09.805462 2191 raft_consensus.cc:2454] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 1.
I20260929 19:06:09.805552 2471 raft_consensus.cc:2454] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 1.
I20260929 19:06:09.805668 2263 leader_election.cc:302] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b9e3bef2dd4e848b166cf89fd9e2f3, 5f41db56342a4bb9999508da3eeb2e9c; no voters:
I20260929 19:06:09.805769 2548 raft_consensus.cc:2781] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:09.806010 2548 raft_consensus.cc:686] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 LEADER]: Becoming Leader. State: Replica: 06b9e3bef2dd4e848b166cf89fd9e2f3, State: Running, Role: LEADER
I20260929 19:06:09.806115 2548 consensus_queue.cc:237] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.806937 2032 catalog_manager.cc:5495] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } health_report { overall_health: UNKNOWN } } }
I20260929 19:06:09.821280 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:09.818885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.26.78.194:41029)
0929 19:06:09.818934 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:09.818935 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:09.818947 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:06:09.819022 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:09.819023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:09.819039 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:09.819098 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:09.819218 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.819718 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.820537 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:09.820741 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:09.820796 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:09.820979 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:09.821075 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:09.821132 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:09.821169 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 19:06:09.821187 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:06:09.823000 2471 raft_consensus.cc:1261] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Refusing update from remote peer 06b9e3bef2dd4e848b166cf89fd9e2f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:09.823040 2191 raft_consensus.cc:1261] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Refusing update from remote peer 06b9e3bef2dd4e848b166cf89fd9e2f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:09.823359 2548 consensus_queue.cc:1035] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:09.823518 2555 consensus_queue.cc:1035] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:09.825879 2558 mvcc.cc:205] Tried to move back new op lower bound from 7334743121192148992 to 7334743121126182912. Current Snapshot: MvccSnapshot[applied={T|T < 7334743121192148992}]
I20260929 19:06:09.826449 2560 mvcc.cc:205] Tried to move back new op lower bound from 7334743121192148992 to 7334743121126182912. Current Snapshot: MvccSnapshot[applied={T|T < 7334743121192148992}]
I20260929 19:06:09.828131 2033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:09.828212 2033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:09.828884 2033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Sending DeleteTablet for 3 replicas of tablet 026645269d7f4a5e909a4519036f61a5
I20260929 19:06:09.829236 2311 tablet_service.cc:1514] Processing DeleteTablet for tablet 026645269d7f4a5e909a4519036f61a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:09 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:38046
I20260929 19:06:09.829463 2567 tablet_replica.cc:330] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: stopping tablet replica
I20260929 19:06:09.829409 2033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 19:06:09.829458 2449 tablet_service.cc:1514] Processing DeleteTablet for tablet 026645269d7f4a5e909a4519036f61a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:09 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:33142
W20260929 19:06:09.829545 2033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:06:09.829639 2153 tablet_service.cc:1514] Processing DeleteTablet for tablet 026645269d7f4a5e909a4519036f61a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:09 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41536
I20260929 19:06:09.829867 2568 tablet_replica.cc:330] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: stopping tablet replica
I20260929 19:06:09.829957 2569 tablet_replica.cc:330] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: stopping tablet replica
I20260929 19:06:09.829977 2568 raft_consensus.cc:2229] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:09.830086 2568 raft_consensus.cc:2258] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:09.830096 2569 raft_consensus.cc:2229] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:09.830513 2569 raft_consensus.cc:2258] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:09.830868 2153 tablet_service.cc:1467] Processing CreateTablet for tablet fbb9440bdf2242318a2373525c7bff6a (DEFAULT_TABLE table=test-table [id=36c443b3dd374b59ae2d7656f95a1298]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:09.831004 2153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb9440bdf2242318a2373525c7bff6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:09.831125 2567 raft_consensus.cc:2229] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:06:09.831238 2567 raft_consensus.cc:2258] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:09.831338 2569 ts_tablet_manager.cc:1905] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:09.832392 2550 tablet_bootstrap.cc:492] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Bootstrap starting.
I20260929 19:06:09.832415 2311 tablet_service.cc:1467] Processing CreateTablet for tablet fbb9440bdf2242318a2373525c7bff6a (DEFAULT_TABLE table=test-table [id=36c443b3dd374b59ae2d7656f95a1298]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:09.832521 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb9440bdf2242318a2373525c7bff6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:09.832825 2550 tablet_bootstrap.cc:654] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:09.833191 2569 ts_tablet_manager.cc:1918] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:09.833312 2569 log.cc:1196] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/026645269d7f4a5e909a4519036f61a5
I20260929 19:06:09.833642 2569 ts_tablet_manager.cc:1939] T 026645269d7f4a5e909a4519036f61a5 P 5f41db56342a4bb9999508da3eeb2e9c: Deleting consensus metadata
I20260929 19:06:09.833709 2550 tablet_bootstrap.cc:492] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: No bootstrap required, opened a new log
I20260929 19:06:09.833765 2550 ts_tablet_manager.cc:1397] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:09.833950 2550 raft_consensus.cc:348] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.834033 2550 raft_consensus.cc:374] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:09.834060 2550 raft_consensus.cc:729] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f41db56342a4bb9999508da3eeb2e9c, State: Initialized, Role: FOLLOWER
I20260929 19:06:09.834043 2449 tablet_service.cc:1467] Processing CreateTablet for tablet fbb9440bdf2242318a2373525c7bff6a (DEFAULT_TABLE table=test-table [id=36c443b3dd374b59ae2d7656f95a1298]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:09.834102 2550 consensus_queue.cc:260] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.834158 2449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb9440bdf2242318a2373525c7bff6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:09.834195 2550 ts_tablet_manager.cc:1428] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:06:09.835606 2547 tablet_bootstrap.cc:492] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Bootstrap starting.
I20260929 19:06:09.835701 2545 tablet_bootstrap.cc:492] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: Bootstrap starting.
I20260929 19:06:09.836022 2547 tablet_bootstrap.cc:654] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:09.836098 2545 tablet_bootstrap.cc:654] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:09.836597 2018 catalog_manager.cc:4841] TS 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615): tablet 026645269d7f4a5e909a4519036f61a5 (table test-table [id=131da00829414245a4e7b86fb6ec9fa4]) successfully deleted
I20260929 19:06:09.836663 2567 ts_tablet_manager.cc:1905] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:09.837119 2568 ts_tablet_manager.cc:1905] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:09.837189 2547 tablet_bootstrap.cc:492] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: No bootstrap required, opened a new log
I20260929 19:06:09.837240 2547 ts_tablet_manager.cc:1397] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:06:09.837423 2547 raft_consensus.cc:348] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.837499 2547 raft_consensus.cc:374] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:09.837527 2547 raft_consensus.cc:729] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 719cfe1acf5e42f8b7d0effaee3bfbb7, State: Initialized, Role: FOLLOWER
I20260929 19:06:09.837570 2547 consensus_queue.cc:260] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.837652 2547 ts_tablet_manager.cc:1428] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:06:09.839227 2568 ts_tablet_manager.cc:1918] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:09.839262 2567 ts_tablet_manager.cc:1918] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:09.839334 2567 log.cc:1196] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/026645269d7f4a5e909a4519036f61a5
I20260929 19:06:09.839346 2568 log.cc:1196] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/026645269d7f4a5e909a4519036f61a5
I20260929 19:06:09.839589 2568 ts_tablet_manager.cc:1939] T 026645269d7f4a5e909a4519036f61a5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting consensus metadata
I20260929 19:06:09.839610 2567 ts_tablet_manager.cc:1939] T 026645269d7f4a5e909a4519036f61a5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting consensus metadata
I20260929 19:06:09.839989 2020 catalog_manager.cc:4841] TS 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029): tablet 026645269d7f4a5e909a4519036f61a5 (table test-table [id=131da00829414245a4e7b86fb6ec9fa4]) successfully deleted
I20260929 19:06:09.839995 2018 catalog_manager.cc:4841] TS 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443): tablet 026645269d7f4a5e909a4519036f61a5 (table test-table [id=131da00829414245a4e7b86fb6ec9fa4]) successfully deleted
I20260929 19:06:09.842242 2545 tablet_bootstrap.cc:492] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: No bootstrap required, opened a new log
I20260929 19:06:09.842308 2545 ts_tablet_manager.cc:1397] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260929 19:06:09.842458 2545 raft_consensus.cc:348] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.842533 2545 raft_consensus.cc:374] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:09.842568 2545 raft_consensus.cc:729] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b9e3bef2dd4e848b166cf89fd9e2f3, State: Initialized, Role: FOLLOWER
I20260929 19:06:09.842618 2545 consensus_queue.cc:260] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:09.842708 2545 ts_tablet_manager.cc:1428] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260929 19:06:09.886683 2238 tablet.cc:2369] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:06:09.960134 2518 tablet.cc:2369] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:06:10.061218 2553 raft_consensus.cc:482] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:06:10.061331 2553 raft_consensus.cc:504] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.061702 2553 leader_election.cc:288] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029), 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443)
I20260929 19:06:10.064682 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.061995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:34207 (local address 127.26.78.194:41029)
0929 19:06:10.062049 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.062051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.062163 (+ 112us) server_negotiation.cc:407] Connection header received
0929 19:06:10.062246 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.062247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.062261 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.062326 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:10.062489 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.062957 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.063961 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.064329 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.064411 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.064420 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.064576 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.064616 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:06:10.064680 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.061985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:57913 (local address 127.26.78.195:42443)
0929 19:06:10.062047 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.062050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.062128 (+ 78us) server_negotiation.cc:407] Connection header received
0929 19:06:10.062205 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.062206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.062217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.062294 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:10.062463 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.062965 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.063976 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.064324 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.064412 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.064420 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.064568 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.064605 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:06:10.064702 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.061915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:42443 (local address 127.26.78.193:57913)
0929 19:06:10.062046 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.062057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.062126 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.062302 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.062303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.062305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:10.062391 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.062393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.063057 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.063058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.063880 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.063883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.064414 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.064415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.064494 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.064497 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.064563 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.064604 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":46,"thread_start_us":67,"threads_started":1}
I20260929 19:06:10.064710 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.061915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41029 (local address 127.26.78.193:34207)
0929 19:06:10.062101 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.062110 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.062171 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.062352 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.062353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.062356 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:10.062420 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.062422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.063048 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.063049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.063863 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.063864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.064412 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.064413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.064489 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.064493 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.064568 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.064610 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":19,"thread_start_us":48,"threads_started":1}
I20260929 19:06:10.064965 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb9440bdf2242318a2373525c7bff6a" candidate_uuid: "5f41db56342a4bb9999508da3eeb2e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" is_pre_election: true
I20260929 19:06:10.065034 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb9440bdf2242318a2373525c7bff6a" candidate_uuid: "5f41db56342a4bb9999508da3eeb2e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" is_pre_election: true
I20260929 19:06:10.065109 2331 raft_consensus.cc:2454] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f41db56342a4bb9999508da3eeb2e9c in term 0.
I20260929 19:06:10.065132 2471 raft_consensus.cc:2454] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f41db56342a4bb9999508da3eeb2e9c in term 0.
I20260929 19:06:10.065280 2124 leader_election.cc:302] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b9e3bef2dd4e848b166cf89fd9e2f3, 5f41db56342a4bb9999508da3eeb2e9c; no voters:
I20260929 19:06:10.065472 2553 raft_consensus.cc:2781] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:06:10.065505 2553 raft_consensus.cc:482] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:06:10.065519 2553 raft_consensus.cc:3037] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.066227 2553 raft_consensus.cc:504] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.066373 2553 leader_election.cc:288] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [CANDIDATE]: Term 1 election: Requested vote from peers 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029), 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443)
I20260929 19:06:10.066529 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb9440bdf2242318a2373525c7bff6a" candidate_uuid: "5f41db56342a4bb9999508da3eeb2e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3"
I20260929 19:06:10.066597 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb9440bdf2242318a2373525c7bff6a" candidate_uuid: "5f41db56342a4bb9999508da3eeb2e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7"
I20260929 19:06:10.066637 2331 raft_consensus.cc:3037] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.066677 2471 raft_consensus.cc:3037] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.067276 2471 raft_consensus.cc:2454] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f41db56342a4bb9999508da3eeb2e9c in term 1.
I20260929 19:06:10.067426 2331 raft_consensus.cc:2454] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f41db56342a4bb9999508da3eeb2e9c in term 1.
I20260929 19:06:10.067479 2122 leader_election.cc:302] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f41db56342a4bb9999508da3eeb2e9c, 719cfe1acf5e42f8b7d0effaee3bfbb7; no voters:
I20260929 19:06:10.067579 2553 raft_consensus.cc:2781] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:10.067699 2553 raft_consensus.cc:686] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 1 LEADER]: Becoming Leader. State: Replica: 5f41db56342a4bb9999508da3eeb2e9c, State: Running, Role: LEADER
I20260929 19:06:10.067785 2553 consensus_queue.cc:237] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.068492 2033 catalog_manager.cc:5495] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193). New cstate: current_term: 1 leader_uuid: "5f41db56342a4bb9999508da3eeb2e9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } health_report { overall_health: UNKNOWN } } }
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708769, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.254@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:06:10.141566 2582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.134576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:37641 (local address 127.0.0.1:59856)
0929 19:06:10.134807 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.134816 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.134879 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.135020 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.135021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.135328 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:10.135377 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.135379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.135993 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.135994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.136518 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.136520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.136818 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.136819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.136858 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.137370 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:10.137374 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:10.138910 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:10.140570 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:10.140577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:10.140579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:10.140778 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:10.140992 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:10.140992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:10.140993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:10.141059 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:10.141281 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:10.141288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:10.141328 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.141377 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.141472 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":127,"threads_started":1}
I20260929 19:06:10.141638 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.134742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.26.78.254:37641)
0929 19:06:10.134844 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.134846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.134860 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:06:10.134935 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.134936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.134945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.135000 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:10.135440 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.135925 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.136588 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.136758 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.136804 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.138996 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:10.138998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:10.138998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:10.139022 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:10.140479 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:10.140864 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:10.140865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:10.140865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:10.140914 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:10.141130 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:10.141131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:10.141131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:10.141227 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:10.141274 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.141402 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.141519 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":52}
I20260929 19:06:10.143709 2033 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:59856:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 19:06:10.143939 2033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:06:10.145442 2153 tablet_service.cc:1467] Processing CreateTablet for tablet bb4c567987f9429080bafb28d607fdc5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=70b3830b151443c5847dc22a8cd94e97]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:06:10.145483 2449 tablet_service.cc:1467] Processing CreateTablet for tablet bb4c567987f9429080bafb28d607fdc5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=70b3830b151443c5847dc22a8cd94e97]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:06:10.145610 2153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb4c567987f9429080bafb28d607fdc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.145645 2449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb4c567987f9429080bafb28d607fdc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.146891 2550 tablet_bootstrap.cc:492] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: Bootstrap starting.
I20260929 19:06:10.146994 2311 tablet_service.cc:1467] Processing CreateTablet for tablet bb4c567987f9429080bafb28d607fdc5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=70b3830b151443c5847dc22a8cd94e97]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:06:10.147147 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb4c567987f9429080bafb28d607fdc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.147328 2550 tablet_bootstrap.cc:654] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.148046 2550 tablet_bootstrap.cc:492] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: No bootstrap required, opened a new log
I20260929 19:06:10.148088 2550 ts_tablet_manager.cc:1397] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:10.148205 2550 raft_consensus.cc:348] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.148262 2550 raft_consensus.cc:374] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.148274 2550 raft_consensus.cc:729] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f41db56342a4bb9999508da3eeb2e9c, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.148347 2550 consensus_queue.cc:260] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.148451 2550 ts_tablet_manager.cc:1428] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:10.148550 2553 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.148623 2545 tablet_bootstrap.cc:492] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Bootstrap starting.
I20260929 19:06:10.148651 2553 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:06:10.148663 2547 tablet_bootstrap.cc:492] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Bootstrap starting.
I20260929 19:06:10.148985 2545 tablet_bootstrap.cc:654] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.149118 2547 tablet_bootstrap.cc:654] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.149722 2545 tablet_bootstrap.cc:492] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: No bootstrap required, opened a new log
I20260929 19:06:10.149765 2545 ts_tablet_manager.cc:1397] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:10.149823 2547 tablet_bootstrap.cc:492] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: No bootstrap required, opened a new log
I20260929 19:06:10.149892 2547 ts_tablet_manager.cc:1397] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:10.149907 2545 raft_consensus.cc:348] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.149973 2545 raft_consensus.cc:374] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.149986 2545 raft_consensus.cc:729] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b9e3bef2dd4e848b166cf89fd9e2f3, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.150048 2545 consensus_queue.cc:260] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.150043 2547 raft_consensus.cc:348] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.150107 2547 raft_consensus.cc:374] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.150139 2547 raft_consensus.cc:729] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 719cfe1acf5e42f8b7d0effaee3bfbb7, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.150153 2545 ts_tablet_manager.cc:1428] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:10.150188 2547 consensus_queue.cc:260] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.150171 2555 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.150264 2555 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:06:10.150286 2547 ts_tablet_manager.cc:1428] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:10.150350 2552 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.150460 2552 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:06:10.376969 2555 raft_consensus.cc:482] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:06:10.377115 2555 raft_consensus.cc:504] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.377319 2555 leader_election.cc:288] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443)
I20260929 19:06:10.377614 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4c567987f9429080bafb28d607fdc5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c" is_pre_election: true
I20260929 19:06:10.377622 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4c567987f9429080bafb28d607fdc5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" is_pre_election: true
I20260929 19:06:10.377728 2471 raft_consensus.cc:2454] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 0.
I20260929 19:06:10.377733 2191 raft_consensus.cc:2454] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 0.
I20260929 19:06:10.377914 2263 leader_election.cc:302] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b9e3bef2dd4e848b166cf89fd9e2f3, 5f41db56342a4bb9999508da3eeb2e9c; no voters:
I20260929 19:06:10.378011 2555 raft_consensus.cc:2781] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:06:10.378046 2555 raft_consensus.cc:482] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:06:10.378072 2555 raft_consensus.cc:3037] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.378798 2555 raft_consensus.cc:504] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.378968 2555 leader_election.cc:288] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 election: Requested vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443)
I20260929 19:06:10.379170 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4c567987f9429080bafb28d607fdc5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c"
I20260929 19:06:10.379189 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4c567987f9429080bafb28d607fdc5" candidate_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7"
I20260929 19:06:10.379261 2191 raft_consensus.cc:3037] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.379261 2471 raft_consensus.cc:3037] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.380018 2191 raft_consensus.cc:2454] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 1.
I20260929 19:06:10.380018 2471 raft_consensus.cc:2454] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b9e3bef2dd4e848b166cf89fd9e2f3 in term 1.
I20260929 19:06:10.380223 2263 leader_election.cc:302] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b9e3bef2dd4e848b166cf89fd9e2f3, 719cfe1acf5e42f8b7d0effaee3bfbb7; no voters:
I20260929 19:06:10.380353 2555 raft_consensus.cc:2781] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:10.380458 2555 raft_consensus.cc:686] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 LEADER]: Becoming Leader. State: Replica: 06b9e3bef2dd4e848b166cf89fd9e2f3, State: Running, Role: LEADER
I20260929 19:06:10.380502 2555 consensus_queue.cc:237] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.380787 2586 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: TxnStatusTablet state changed. Reason: New leader 06b9e3bef2dd4e848b166cf89fd9e2f3. Latest consensus state: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.380914 2586 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:06:10.381067 2588 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 19:06:10.381071 2032 catalog_manager.cc:5495] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } health_report { overall_health: UNKNOWN } } }
I20260929 19:06:10.381192 2588 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 19:06:10.433902 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.431325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38064 (local address 127.26.78.194:41029)
0929 19:06:10.431402 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.431404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.431413 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:10.431477 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.431478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.431490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.431568 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:10.431862 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.432359 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.433161 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.433352 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.433427 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.433611 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.433654 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.433714 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.433780 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.433801 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:06:10.433924 2582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.431148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41029 (local address 127.0.0.1:38064)
0929 19:06:10.431270 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.431283 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.431398 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.431579 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.431579 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.431580 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:06:10.431746 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.431751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.432454 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.432455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.433051 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.433053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.433427 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.433428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.433500 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.433517 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.433718 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.433728 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.433776 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.433850 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 19:06:10.435288 2471 raft_consensus.cc:1261] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Refusing update from remote peer 06b9e3bef2dd4e848b166cf89fd9e2f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.435319 2191 raft_consensus.cc:1261] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Refusing update from remote peer 06b9e3bef2dd4e848b166cf89fd9e2f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.435567 2586 consensus_queue.cc:1035] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.435678 2555 consensus_queue.cc:1035] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.436149 2552 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: TxnStatusTablet state changed. Reason: New leader 06b9e3bef2dd4e848b166cf89fd9e2f3. Latest consensus state: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.436250 2552 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:06:10.436332 2553 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: TxnStatusTablet state changed. Reason: New leader 06b9e3bef2dd4e848b166cf89fd9e2f3. Latest consensus state: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.436415 2553 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:06:10.437554 2555 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.437561 2586 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.437647 2555 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:06:10.437669 2586 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 06b9e3bef2dd4e848b166cf89fd9e2f3: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:06:10.438642 2553 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.438727 2553 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 5f41db56342a4bb9999508da3eeb2e9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:06:10.438995 2552 tablet_replica.cc:439] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } } }
I20260929 19:06:10.439088 2552 tablet_replica.cc:442] T bb4c567987f9429080bafb28d607fdc5 P 719cfe1acf5e42f8b7d0effaee3bfbb7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:06:10.442893 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.439913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.26.78.193:37615)
0929 19:06:10.439998 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.440000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.440012 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:06:10.440106 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.440107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.440124 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.440192 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:10.440388 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.440885 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.442011 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.442267 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.442326 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.442513 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.442612 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.442664 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.442755 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.442772 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67,"thread_start_us":48,"threads_started":1}
I20260929 19:06:10.448344 2605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.445432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41029 (local address 127.26.78.193:46179)
0929 19:06:10.445584 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.445595 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.445657 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.445789 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.445790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.445791 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:06:10.445880 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.445882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.446788 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.446789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.447431 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.447433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.447917 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.447918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.447968 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.447982 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.448184 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.448186 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.448220 (+ 34us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.448255 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":98,"threads_started":1}
I20260929 19:06:10.448334 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.445508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:46179 (local address 127.26.78.194:41029)
0929 19:06:10.445559 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.445561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.445675 (+ 114us) server_negotiation.cc:407] Connection header received
0929 19:06:10.445703 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.445704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.445716 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.445782 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:10.445963 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.446699 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.447514 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.447793 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.447872 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.448072 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.448106 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.448175 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.448226 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.448246 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:06:10.450857 2471 raft_consensus.cc:1261] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Refusing update from remote peer 5f41db56342a4bb9999508da3eeb2e9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.450922 2331 raft_consensus.cc:1261] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Refusing update from remote peer 5f41db56342a4bb9999508da3eeb2e9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.451130 2575 consensus_queue.cc:1035] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.451297 2553 consensus_queue.cc:1035] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.453056 2592 mvcc.cc:205] Tried to move back new op lower bound from 7334743123764858880 to 7334743122198188032. Current Snapshot: MvccSnapshot[applied={T|T < 7334743123764858880}]
I20260929 19:06:10.460134 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.457572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:52773 (local address 127.26.78.193:37615)
0929 19:06:10.457667 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.457669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.457677 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:10.457734 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.457735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.457745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.457799 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:10.457939 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.458402 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.459357 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.459642 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.459683 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.459879 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.459912 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.459991 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.460042 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.460064 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 19:06:10.460391 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.457391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:37615 (local address 127.26.78.194:52773)
0929 19:06:10.457618 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.457630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.457682 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.457803 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.457803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.457805 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:06:10.457873 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.457874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.458487 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.458488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.459159 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.459161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.459743 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.459744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.459786 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.459797 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.459995 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.459997 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.460042 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.460088 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
I20260929 19:06:10.473621 2032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:10.473742 2032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:10.474433 2032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Sending DeleteTablet for 3 replicas of tablet fbb9440bdf2242318a2373525c7bff6a
I20260929 19:06:10.474782 2153 tablet_service.cc:1514] Processing DeleteTablet for tablet fbb9440bdf2242318a2373525c7bff6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41536
I20260929 19:06:10.475009 2449 tablet_service.cc:1514] Processing DeleteTablet for tablet fbb9440bdf2242318a2373525c7bff6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:33142
I20260929 19:06:10.475041 2613 tablet_replica.cc:330] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: stopping tablet replica
I20260929 19:06:10.475225 2613 raft_consensus.cc:2229] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:06:10.475332 2311 tablet_service.cc:1514] Processing DeleteTablet for tablet fbb9440bdf2242318a2373525c7bff6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:38046
I20260929 19:06:10.475358 2613 raft_consensus.cc:2258] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.475708 2613 ts_tablet_manager.cc:1905] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.477532 2613 ts_tablet_manager.cc:1918] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:06:10.477666 2613 log.cc:1196] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/fbb9440bdf2242318a2373525c7bff6a
I20260929 19:06:10.478025 2613 ts_tablet_manager.cc:1939] T fbb9440bdf2242318a2373525c7bff6a P 5f41db56342a4bb9999508da3eeb2e9c: Deleting consensus metadata
I20260929 19:06:10.478411 2018 catalog_manager.cc:4841] TS 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615): tablet fbb9440bdf2242318a2373525c7bff6a (table test-table [id=36c443b3dd374b59ae2d7656f95a1298]) successfully deleted
I20260929 19:06:10.481318 2615 tablet_replica.cc:330] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: stopping tablet replica
I20260929 19:06:10.481397 2615 raft_consensus.cc:2229] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:10.481439 2615 raft_consensus.cc:2258] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.481907 2615 ts_tablet_manager.cc:1905] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.484249 2614 tablet_replica.cc:330] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: stopping tablet replica
I20260929 19:06:10.484370 2614 raft_consensus.cc:2229] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:10.484462 2614 raft_consensus.cc:2258] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.484756 2614 ts_tablet_manager.cc:1905] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.486392 2614 ts_tablet_manager.cc:1918] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:06:10.483552 2615 ts_tablet_manager.cc:1918] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:06:10.486567 2615 log.cc:1196] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/fbb9440bdf2242318a2373525c7bff6a
I20260929 19:06:10.486871 2615 ts_tablet_manager.cc:1939] T fbb9440bdf2242318a2373525c7bff6a P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting consensus metadata
I20260929 19:06:10.487141 2614 log.cc:1196] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/fbb9440bdf2242318a2373525c7bff6a
I20260929 19:06:10.487259 2020 catalog_manager.cc:4841] TS 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029): tablet fbb9440bdf2242318a2373525c7bff6a (table test-table [id=36c443b3dd374b59ae2d7656f95a1298]) successfully deleted
I20260929 19:06:10.487370 2614 ts_tablet_manager.cc:1939] T fbb9440bdf2242318a2373525c7bff6a P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting consensus metadata
I20260929 19:06:10.487708 2018 catalog_manager.cc:4841] TS 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443): tablet fbb9440bdf2242318a2373525c7bff6a (table test-table [id=36c443b3dd374b59ae2d7656f95a1298]) successfully deleted
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[1988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 19:06:10.510721 2083 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:10.503499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.26.78.254:37641)
0929 19:06:10.503547 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.503549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.503709 (+ 160us) server_negotiation.cc:407] Connection header received
0929 19:06:10.503774 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.503775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.503789 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.503864 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:10.504562 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.505405 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.506550 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.506877 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.506954 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.510644 (+ 3690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59864: BlockingRecv error: recv got EOF from 127.0.0.1:59864 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[1988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM
I20260929 19:06:10.521037 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.512209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59866 (local address 127.26.78.254:37641)
0929 19:06:10.512257 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.512259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.512394 (+ 135us) server_negotiation.cc:407] Connection header received
0929 19:06:10.512451 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.512452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.512465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.512519 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:10.513171 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.513914 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.514978 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.515313 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.515372 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.518204 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:10.518206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:10.518206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:10.518214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:10.519822 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:10.520244 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:10.520245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:10.520246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:10.520300 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:10.520477 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:10.520478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:10.520478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:10.520592 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:10.520657 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.520848 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.520925 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:06:10.523331 2032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:06:10.523509 2032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:06:10.525003 2449 tablet_service.cc:1467] Processing CreateTablet for tablet fc2682f491304fe1a408154f82541056 (DEFAULT_TABLE table=test-table [id=acb855ba02694a86977e3e5abda45d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:10.525148 2449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2682f491304fe1a408154f82541056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.525120 2311 tablet_service.cc:1467] Processing CreateTablet for tablet fc2682f491304fe1a408154f82541056 (DEFAULT_TABLE table=test-table [id=acb855ba02694a86977e3e5abda45d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:10.525249 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2682f491304fe1a408154f82541056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.526331 2547 tablet_bootstrap.cc:492] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Bootstrap starting.
I20260929 19:06:10.526371 2153 tablet_service.cc:1467] Processing CreateTablet for tablet fc2682f491304fe1a408154f82541056 (DEFAULT_TABLE table=test-table [id=acb855ba02694a86977e3e5abda45d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:10.526486 2153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2682f491304fe1a408154f82541056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.526614 2545 tablet_bootstrap.cc:492] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Bootstrap starting.
I20260929 19:06:10.526777 2547 tablet_bootstrap.cc:654] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.526909 2545 tablet_bootstrap.cc:654] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.527607 2545 tablet_bootstrap.cc:492] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: No bootstrap required, opened a new log
I20260929 19:06:10.527664 2545 ts_tablet_manager.cc:1397] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 19:06:10.527765 2545 raft_consensus.cc:348] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.527856 2545 raft_consensus.cc:374] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.527878 2545 raft_consensus.cc:729] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b9e3bef2dd4e848b166cf89fd9e2f3, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.527925 2545 consensus_queue.cc:260] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.528012 2545 ts_tablet_manager.cc:1428] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:10.528106 2550 tablet_bootstrap.cc:492] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: Bootstrap starting.
I20260929 19:06:10.528353 2547 tablet_bootstrap.cc:492] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: No bootstrap required, opened a new log
I20260929 19:06:10.528422 2547 ts_tablet_manager.cc:1397] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:06:10.528517 2550 tablet_bootstrap.cc:654] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.528589 2547 raft_consensus.cc:348] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.528661 2547 raft_consensus.cc:374] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.528699 2547 raft_consensus.cc:729] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 719cfe1acf5e42f8b7d0effaee3bfbb7, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.528748 2547 consensus_queue.cc:260] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.528851 2547 ts_tablet_manager.cc:1428] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:10.529587 2550 tablet_bootstrap.cc:492] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: No bootstrap required, opened a new log
I20260929 19:06:10.529640 2550 ts_tablet_manager.cc:1397] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:06:10.529785 2550 raft_consensus.cc:348] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.529898 2550 raft_consensus.cc:374] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.529927 2550 raft_consensus.cc:729] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f41db56342a4bb9999508da3eeb2e9c, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.529975 2550 consensus_queue.cc:260] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.530079 2550 ts_tablet_manager.cc:1428] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:10.630265 2552 raft_consensus.cc:482] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:06:10.630386 2552 raft_consensus.cc:504] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.630779 2552 leader_election.cc:288] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029)
I20260929 19:06:10.633909 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.631142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:47617 (local address 127.26.78.194:41029)
0929 19:06:10.631193 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.631195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.631342 (+ 147us) server_negotiation.cc:407] Connection header received
0929 19:06:10.631419 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.631420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.631430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.631491 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:10.631671 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.632118 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.633129 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.633506 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.633591 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.633600 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.633776 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.633844 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:06:10.633915 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.631081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:54463 (local address 127.26.78.193:37615)
0929 19:06:10.631126 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.631128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.631182 (+ 54us) server_negotiation.cc:407] Connection header received
0929 19:06:10.631316 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.631318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.631328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.631397 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:10.631557 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.632106 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.633126 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.633504 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.633595 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.633605 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.633732 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.633784 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:06:10.633915 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.631069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41029 (local address 127.26.78.195:47617)
0929 19:06:10.631226 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.631235 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.631327 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.631506 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.631507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.631509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:10.631598 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.631600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.632245 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.632246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.633029 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.633031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.633617 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.633618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.633681 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.633684 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.633760 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.633800 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260929 19:06:10.634107 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.630975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:37615 (local address 127.26.78.195:54463)
0929 19:06:10.631115 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.631126 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.631215 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.631418 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.631419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.631422 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:10.631485 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.631487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.632197 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.632198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.633031 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.633033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.633595 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.633596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.633661 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.633665 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.633707 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.633760 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":40,"thread_start_us":66,"threads_started":1}
I20260929 19:06:10.634423 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2682f491304fe1a408154f82541056" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" is_pre_election: true
I20260929 19:06:10.634523 2331 raft_consensus.cc:2454] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 0.
I20260929 19:06:10.634459 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2682f491304fe1a408154f82541056" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c" is_pre_election: true
I20260929 19:06:10.634574 2191 raft_consensus.cc:2454] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 0.
I20260929 19:06:10.634756 2403 leader_election.cc:302] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f41db56342a4bb9999508da3eeb2e9c, 719cfe1acf5e42f8b7d0effaee3bfbb7; no voters:
I20260929 19:06:10.634905 2552 raft_consensus.cc:2781] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:06:10.634950 2552 raft_consensus.cc:482] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:06:10.634966 2552 raft_consensus.cc:3037] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.635658 2552 raft_consensus.cc:504] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.635804 2552 leader_election.cc:288] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 election: Requested vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029)
I20260929 19:06:10.635928 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2682f491304fe1a408154f82541056" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c"
I20260929 19:06:10.635947 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2682f491304fe1a408154f82541056" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3"
I20260929 19:06:10.636000 2191 raft_consensus.cc:3037] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.636031 2331 raft_consensus.cc:3037] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.636629 2191 raft_consensus.cc:2454] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 1.
I20260929 19:06:10.636704 2331 raft_consensus.cc:2454] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 1.
I20260929 19:06:10.636795 2403 leader_election.cc:302] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f41db56342a4bb9999508da3eeb2e9c, 719cfe1acf5e42f8b7d0effaee3bfbb7; no voters:
I20260929 19:06:10.636904 2552 raft_consensus.cc:2781] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:10.637053 2552 raft_consensus.cc:686] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 LEADER]: Becoming Leader. State: Replica: 719cfe1acf5e42f8b7d0effaee3bfbb7, State: Running, Role: LEADER
I20260929 19:06:10.637109 2552 consensus_queue.cc:237] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.637964 2032 catalog_manager.cc:5495] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } health_report { overall_health: HEALTHY } } }
I20260929 19:06:10.646574 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.644136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.26.78.195:42443)
0929 19:06:10.644297 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.644299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.644306 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:06:10.644344 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.644344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.644362 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.644434 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:10.644581 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.645083 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.645805 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.646060 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.646112 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.646269 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.646374 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.646428 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.646464 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.646479 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260929 19:06:10.647531 2331 raft_consensus.cc:1261] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Refusing update from remote peer 719cfe1acf5e42f8b7d0effaee3bfbb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.647567 2191 raft_consensus.cc:1261] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Refusing update from remote peer 719cfe1acf5e42f8b7d0effaee3bfbb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.647797 2633 consensus_queue.cc:1035] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.647965 2552 consensus_queue.cc:1035] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.651566 2032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:10.651662 2032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:10.652302 2032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Sending DeleteTablet for 3 replicas of tablet fc2682f491304fe1a408154f82541056
I20260929 19:06:10.652566 2311 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2682f491304fe1a408154f82541056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:38046
I20260929 19:06:10.652678 2153 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2682f491304fe1a408154f82541056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41536
I20260929 19:06:10.652669 2449 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2682f491304fe1a408154f82541056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:33142
I20260929 19:06:10.652770 2613 tablet_replica.cc:330] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: stopping tablet replica
I20260929 19:06:10.652809 2615 tablet_replica.cc:330] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: stopping tablet replica
I20260929 19:06:10.652820 2613 raft_consensus.cc:2229] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:10.652874 2613 raft_consensus.cc:2258] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.652863 2615 raft_consensus.cc:2229] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:10.652925 2615 raft_consensus.cc:2258] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.652989 2614 tablet_replica.cc:330] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: stopping tablet replica
I20260929 19:06:10.653034 2614 raft_consensus.cc:2229] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:06:10.653020 2032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 19:06:10.653112 2614 raft_consensus.cc:2258] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 19:06:10.653172 2032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:06:10.653456 2614 ts_tablet_manager.cc:1905] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.653576 2615 ts_tablet_manager.cc:1905] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.653683 2613 ts_tablet_manager.cc:1905] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.655288 2613 ts_tablet_manager.cc:1918] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:10.655340 2615 ts_tablet_manager.cc:1918] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:10.655356 2613 log.cc:1196] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/fc2682f491304fe1a408154f82541056
I20260929 19:06:10.655390 2615 log.cc:1196] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/fc2682f491304fe1a408154f82541056
I20260929 19:06:10.655581 2613 ts_tablet_manager.cc:1939] T fc2682f491304fe1a408154f82541056 P 5f41db56342a4bb9999508da3eeb2e9c: Deleting consensus metadata
I20260929 19:06:10.655598 2615 ts_tablet_manager.cc:1939] T fc2682f491304fe1a408154f82541056 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting consensus metadata
I20260929 19:06:10.656208 2614 ts_tablet_manager.cc:1918] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:10.656229 2449 tablet_service.cc:1467] Processing CreateTablet for tablet 533b72e2ca674b36b1bbd979135f0e78 (DEFAULT_TABLE table=test-table [id=a005ebf02094415793aa70a2c83d2cca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:10.656270 2614 log.cc:1196] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/fc2682f491304fe1a408154f82541056
I20260929 19:06:10.656288 2153 tablet_service.cc:1467] Processing CreateTablet for tablet 533b72e2ca674b36b1bbd979135f0e78 (DEFAULT_TABLE table=test-table [id=a005ebf02094415793aa70a2c83d2cca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:10.656335 2449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 533b72e2ca674b36b1bbd979135f0e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.656395 2153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 533b72e2ca674b36b1bbd979135f0e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.656490 2614 ts_tablet_manager.cc:1939] T fc2682f491304fe1a408154f82541056 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting consensus metadata
I20260929 19:06:10.656757 2311 tablet_service.cc:1467] Processing CreateTablet for tablet 533b72e2ca674b36b1bbd979135f0e78 (DEFAULT_TABLE table=test-table [id=a005ebf02094415793aa70a2c83d2cca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:10.656849 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 533b72e2ca674b36b1bbd979135f0e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:10.657497 2547 tablet_bootstrap.cc:492] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Bootstrap starting.
I20260929 19:06:10.657933 2547 tablet_bootstrap.cc:654] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.658061 2545 tablet_bootstrap.cc:492] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Bootstrap starting.
I20260929 19:06:10.658283 2550 tablet_bootstrap.cc:492] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c: Bootstrap starting.
I20260929 19:06:10.658355 2545 tablet_bootstrap.cc:654] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.658624 2550 tablet_bootstrap.cc:654] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:10.658784 2020 catalog_manager.cc:4841] TS 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029): tablet fc2682f491304fe1a408154f82541056 (table test-table [id=acb855ba02694a86977e3e5abda45d39]) successfully deleted
I20260929 19:06:10.658967 2018 catalog_manager.cc:4841] TS 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443): tablet fc2682f491304fe1a408154f82541056 (table test-table [id=acb855ba02694a86977e3e5abda45d39]) successfully deleted
I20260929 19:06:10.659226 2545 tablet_bootstrap.cc:492] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: No bootstrap required, opened a new log
I20260929 19:06:10.659276 2545 ts_tablet_manager.cc:1397] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:10.659269 2547 tablet_bootstrap.cc:492] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: No bootstrap required, opened a new log
I20260929 19:06:10.659358 2547 ts_tablet_manager.cc:1397] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:06:10.659430 2545 raft_consensus.cc:348] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.659509 2545 raft_consensus.cc:374] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.659534 2545 raft_consensus.cc:729] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b9e3bef2dd4e848b166cf89fd9e2f3, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.659535 2547 raft_consensus.cc:348] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.659584 2545 consensus_queue.cc:260] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.659618 2547 raft_consensus.cc:374] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.659642 2547 raft_consensus.cc:729] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 719cfe1acf5e42f8b7d0effaee3bfbb7, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.659685 2545 ts_tablet_manager.cc:1428] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:10.659685 2547 consensus_queue.cc:260] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.659770 2547 ts_tablet_manager.cc:1428] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:10.660148 2018 catalog_manager.cc:4841] TS 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615): tablet fc2682f491304fe1a408154f82541056 (table test-table [id=acb855ba02694a86977e3e5abda45d39]) successfully deleted
I20260929 19:06:10.660743 2550 tablet_bootstrap.cc:492] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c: No bootstrap required, opened a new log
I20260929 19:06:10.660797 2550 ts_tablet_manager.cc:1397] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 19:06:10.660899 2550 raft_consensus.cc:348] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.660976 2550 raft_consensus.cc:374] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:10.660997 2550 raft_consensus.cc:729] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f41db56342a4bb9999508da3eeb2e9c, State: Initialized, Role: FOLLOWER
I20260929 19:06:10.661047 2550 consensus_queue.cc:260] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.661140 2550 ts_tablet_manager.cc:1428] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:10.753429 2633 raft_consensus.cc:482] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:06:10.753543 2633 raft_consensus.cc:504] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.753733 2633 leader_election.cc:288] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029)
I20260929 19:06:10.753986 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533b72e2ca674b36b1bbd979135f0e78" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" is_pre_election: true
I20260929 19:06:10.753953 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533b72e2ca674b36b1bbd979135f0e78" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c" is_pre_election: true
I20260929 19:06:10.754108 2331 raft_consensus.cc:2454] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 0.
I20260929 19:06:10.754140 2191 raft_consensus.cc:2454] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 0.
I20260929 19:06:10.754330 2405 leader_election.cc:302] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b9e3bef2dd4e848b166cf89fd9e2f3, 719cfe1acf5e42f8b7d0effaee3bfbb7; no voters:
I20260929 19:06:10.754451 2633 raft_consensus.cc:2781] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:06:10.754504 2633 raft_consensus.cc:482] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:06:10.754525 2633 raft_consensus.cc:3037] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.755519 2633 raft_consensus.cc:504] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.755653 2633 leader_election.cc:288] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 election: Requested vote from peers 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615), 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029)
I20260929 19:06:10.755878 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533b72e2ca674b36b1bbd979135f0e78" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3"
I20260929 19:06:10.755880 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533b72e2ca674b36b1bbd979135f0e78" candidate_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f41db56342a4bb9999508da3eeb2e9c"
I20260929 19:06:10.755954 2191 raft_consensus.cc:3037] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.755972 2331 raft_consensus.cc:3037] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:10.756664 2331 raft_consensus.cc:2454] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 1.
I20260929 19:06:10.756680 2191 raft_consensus.cc:2454] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 719cfe1acf5e42f8b7d0effaee3bfbb7 in term 1.
I20260929 19:06:10.756842 2403 leader_election.cc:302] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f41db56342a4bb9999508da3eeb2e9c, 719cfe1acf5e42f8b7d0effaee3bfbb7; no voters:
I20260929 19:06:10.756978 2633 raft_consensus.cc:2781] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:10.757040 2633 raft_consensus.cc:686] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 LEADER]: Becoming Leader. State: Replica: 719cfe1acf5e42f8b7d0effaee3bfbb7, State: Running, Role: LEADER
I20260929 19:06:10.757086 2633 consensus_queue.cc:237] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } }
I20260929 19:06:10.757714 2032 catalog_manager.cc:5495] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "719cfe1acf5e42f8b7d0effaee3bfbb7" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42443 } health_report { overall_health: HEALTHY } } }
I20260929 19:06:10.783890 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.781196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:41029 (local address 127.26.78.195:32935)
0929 19:06:10.781393 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.781405 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.781495 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.781647 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.781648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.781650 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:06:10.781710 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.781711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.782446 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.782448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.783045 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.783046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.783417 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.783418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.783487 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.783511 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.783704 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.783706 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.783755 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.783810 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260929 19:06:10.783864 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.781281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:32935 (local address 127.26.78.194:41029)
0929 19:06:10.781370 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.781373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.781485 (+ 112us) server_negotiation.cc:407] Connection header received
0929 19:06:10.781564 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.781565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.781575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.781643 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:10.781866 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.782360 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.783136 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.783342 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.783419 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.783597 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.783639 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.783701 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.783762 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.783784 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 19:06:10.786291 2331 raft_consensus.cc:1261] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Refusing update from remote peer 719cfe1acf5e42f8b7d0effaee3bfbb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.786319 2191 raft_consensus.cc:1261] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Refusing update from remote peer 719cfe1acf5e42f8b7d0effaee3bfbb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:10.786538 2640 consensus_queue.cc:1035] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f41db56342a4bb9999508da3eeb2e9c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.786636 2633 consensus_queue.cc:1035] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b9e3bef2dd4e848b166cf89fd9e2f3" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 41029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:10.793783 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.791496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:53573 (local address 127.26.78.195:42443)
0929 19:06:10.791546 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:10.791548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:10.791556 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:10.791581 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:10.791582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:10.791592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:10.791677 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:10.791794 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.792298 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.793057 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.793344 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.793419 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.793541 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.793578 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.793629 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:10.793694 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 19:06:10.793715 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:06:10.793840 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.791339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:42443 (local address 127.26.78.194:53573)
0929 19:06:10.791409 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.791421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:10.791463 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:10.791679 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:10.791680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:10.791682 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:06:10.791726 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.791727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.792392 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.792393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.792971 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:10.792973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:10.793418 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:10.793419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:10.793477 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:10.793484 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:06:10.793631 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:06:10.793634 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:10.793681 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:06:10.793737 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 19:06:10.799618 2032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:10.799736 2032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:10.800405 2032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c931ed6eac8b4b19a1be17697d03e5ee: Sending DeleteTablet for 3 replicas of tablet 533b72e2ca674b36b1bbd979135f0e78
I20260929 19:06:10.800642 2311 tablet_service.cc:1514] Processing DeleteTablet for tablet 533b72e2ca674b36b1bbd979135f0e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:38046
I20260929 19:06:10.800665 2153 tablet_service.cc:1514] Processing DeleteTablet for tablet 533b72e2ca674b36b1bbd979135f0e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41536
I20260929 19:06:10.800760 2613 tablet_replica.cc:330] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c: stopping tablet replica
I20260929 19:06:10.800757 2449 tablet_service.cc:1514] Processing DeleteTablet for tablet 533b72e2ca674b36b1bbd979135f0e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:10 UTC) from {username='oryx', principal='oryx/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:33142
I20260929 19:06:10.800781 2615 tablet_replica.cc:330] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: stopping tablet replica
I20260929 19:06:10.800806 2613 raft_consensus.cc:2229] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:10.800840 2614 tablet_replica.cc:330] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: stopping tablet replica
I20260929 19:06:10.800839 2615 raft_consensus.cc:2229] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:10.800845 2613 raft_consensus.cc:2258] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.800874 2615 raft_consensus.cc:2258] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.800877 2614 raft_consensus.cc:2229] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:06:10.800940 2614 raft_consensus.cc:2258] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:10.801151 2613 ts_tablet_manager.cc:1905] T 533b72e2ca674b36b1bbd979135f0e78 P 5f41db56342a4bb9999508da3eeb2e9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.801249 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 2105
I20260929 19:06:10.801278 2614 ts_tablet_manager.cc:1905] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.803210 2614 ts_tablet_manager.cc:1918] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:06:10.803280 2614 log.cc:1196] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/533b72e2ca674b36b1bbd979135f0e78
I20260929 19:06:10.803496 2614 ts_tablet_manager.cc:1939] T 533b72e2ca674b36b1bbd979135f0e78 P 719cfe1acf5e42f8b7d0effaee3bfbb7: Deleting consensus metadata
I20260929 19:06:10.803964 2018 catalog_manager.cc:4841] TS 719cfe1acf5e42f8b7d0effaee3bfbb7 (127.26.78.195:42443): tablet 533b72e2ca674b36b1bbd979135f0e78 (table test-table [id=a005ebf02094415793aa70a2c83d2cca]) successfully deleted
I20260929 19:06:10.804661 2615 ts_tablet_manager.cc:1905] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:10.806622 2615 ts_tablet_manager.cc:1918] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:06:10.806694 2615 log.cc:1196] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/533b72e2ca674b36b1bbd979135f0e78
I20260929 19:06:10.806914 2615 ts_tablet_manager.cc:1939] T 533b72e2ca674b36b1bbd979135f0e78 P 06b9e3bef2dd4e848b166cf89fd9e2f3: Deleting consensus metadata
I20260929 19:06:10.807683 2020 catalog_manager.cc:4841] TS 06b9e3bef2dd4e848b166cf89fd9e2f3 (127.26.78.194:41029): tablet 533b72e2ca674b36b1bbd979135f0e78 (table test-table [id=a005ebf02094415793aa70a2c83d2cca]) successfully deleted
W20260929 19:06:10.809770 2018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:37615 (error 108)
I20260929 19:06:10.810106 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 2246
I20260929 19:06:10.810949 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:10.810193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:37615 (local address 127.0.0.1:41544)
0929 19:06:10.810851 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:10.810898 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:37615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":69,"threads_started":1}
W20260929 19:06:10.811239 2018 catalog_manager.cc:4584] TS 5f41db56342a4bb9999508da3eeb2e9c (127.26.78.193:37615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 533b72e2ca674b36b1bbd979135f0e78: Network error: Client connection negotiation failed: client connection to 127.26.78.193:37615: connect: Connection refused (error 111)
I20260929 19:06:10.816800 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 2386
I20260929 19:06:10.822645 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 2001
2026-09-29T19:06:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2938 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[2655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[2655](info): set up 2 sockets
Sep 29 19:06:10 dist-test-slave-xz51 krb5kdc[2655](info): commencing operation
krb5kdc: starting...
W20260929 19:06:11.866468 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:06:11 dist-test-slave-xz51 krb5kdc[2655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:06:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:06:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:06:11.984463 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33295
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:33295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:12.067298 2668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:12.067557 2668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:12.067597 2668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:12.068799 2668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:06:12.068871 2668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:12.068894 2668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:12.068915 2668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:06:12.068944 2668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:06:12.070546 2668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:33295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:12.070853 2668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:12.071092 2668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:12.076185 2673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:12.076285 2674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:12.076185 2676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:12.076447 2668 server_base.cc:1034] running on GCE node
I20260929 19:06:12.076681 2668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:12.077028 2668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:12.078213 2668 hybrid_clock.cc:648] HybridClock initialized: now 1790708772078193 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T19:06:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1258 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:06:12 dist-test-slave-xz51 krb5kdc[2681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:06:12 dist-test-slave-xz51 krb5kdc[2681](info): set up 2 sockets
Sep 29 19:06:12 dist-test-slave-xz51 krb5kdc[2681](info): commencing operation
krb5kdc: starting...
W20260929 19:06:13.124293 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:06:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:06:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:06:13.245031 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33151
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:33151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:13.326812 2694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:13.327060 2694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:13.327098 2694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:13.328262 2694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:06:13.328337 2694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:13.328361 2694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:13.328382 2694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:06:13.328410 2694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:06:13.329977 2694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:33151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:13.330257 2694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:13.330487 2694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:13.335515 2699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:13.335580 2694 server_base.cc:1034] running on GCE node
W20260929 19:06:13.335533 2702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:13.335521 2700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:13.335994 2694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:13.336315 2694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:13.337493 2694 hybrid_clock.cc:648] HybridClock initialized: now 1790708773337491 us; error 37 us; skew 500 ppm
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:13.344399 2694 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:06:13.345000 2694 webserver.cc:460] Webserver started at http://127.26.78.254:35105/ using document root <none> and password file <none>
I20260929 19:06:13.345286 2694 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:13.345368 2694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:13.345549 2694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:13.346843 2694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "e6edcb8f80c64d41848fd6336e13cb0a"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "03eea8ebc20f6bda099b41cef1285d1f"
server_key_iv: "b6313567a3c527b2bdfbdda75d9bc9dc"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.347254 2694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "e6edcb8f80c64d41848fd6336e13cb0a"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "03eea8ebc20f6bda099b41cef1285d1f"
server_key_iv: "b6313567a3c527b2bdfbdda75d9bc9dc"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.348807 2694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:06:13.349666 2709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.349920 2694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:13.350026 2694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "e6edcb8f80c64d41848fd6336e13cb0a"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "03eea8ebc20f6bda099b41cef1285d1f"
server_key_iv: "b6313567a3c527b2bdfbdda75d9bc9dc"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.350113 2694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:13.359925 2694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:13.361886 2694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:13.362056 2694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:13.366041 2694 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:33151
I20260929 19:06:13.366071 2761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:33151 every 8 connection(s)
I20260929 19:06:13.366433 2694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:06:13.367287 2762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:13.369530 2762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Bootstrap starting.
I20260929 19:06:13.369974 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 2694
I20260929 19:06:13.370096 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:06:13.370314 2762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:13.370316 26939 external_mini_cluster.cc:1428] Setting key 29c482c1e82541f023b16be4db027735
I20260929 19:06:13.370654 2762 log.cc:824] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:13.371470 2762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: No bootstrap required, opened a new log
I20260929 19:06:13.373927 2762 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:13.374176 2762 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:13.374222 2762 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6edcb8f80c64d41848fd6336e13cb0a, State: Initialized, Role: FOLLOWER
I20260929 19:06:13.374356 2762 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:13.374449 2762 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:06:13.374493 2762 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:06:13.374542 2762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:13.375121 2762 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:13.375224 2762 leader_election.cc:302] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6edcb8f80c64d41848fd6336e13cb0a; no voters:
I20260929 19:06:13.375396 2762 leader_election.cc:288] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:06:13.375468 2767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:13.375620 2767 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 LEADER]: Becoming Leader. State: Replica: e6edcb8f80c64d41848fd6336e13cb0a, State: Running, Role: LEADER
I20260929 19:06:13.375771 2767 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:13.375631 2762 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:06:13.376238 2768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } } }
I20260929 19:06:13.376356 2768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:13.376237 2769 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6edcb8f80c64d41848fd6336e13cb0a. Latest consensus state: current_term: 1 leader_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } } }
I20260929 19:06:13.376549 2769 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:13.376660 2771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:06:13.377441 2771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:13.379601 2771 catalog_manager.cc:1347] Generated new cluster ID: 5c2a85358e7c4184ba0cd60a48f68658
I20260929 19:06:13.379650 2771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:06:13.382471 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:13.370792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.26.78.254:33151)
0929 19:06:13.370949 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:13.370959 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:13.370995 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:06:13.371627 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:13.371636 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:13.371698 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:13.371908 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:13.372502 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.373988 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.375047 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.375673 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.375779 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:13.378659 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:13.378663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:13.378663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:13.378675 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:13.380739 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.381146 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.381149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.381149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.381265 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.381485 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.381486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.381486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.381814 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:13.381999 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:13.382092 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 19:06:13.382234 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260929 19:06:13.403661 2771 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:06:13.404277 2771 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:06:13.414829 2771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Generated new TSK 0
I20260929 19:06:13.415040 2771 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:06:13.449489 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:33151
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:13.536377 2789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:13.536579 2789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:13.536602 2789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:13.537684 2789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:13.537719 2789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:13.537744 2789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:06:13.539109 2789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:33151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:13.539294 2789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:13.539471 2789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:13.544536 2794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:13.544531 2795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:13.544552 2789 server_base.cc:1034] running on GCE node
W20260929 19:06:13.544536 2797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:13.544936 2789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:13.545229 2789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:13.546392 2789 hybrid_clock.cc:648] HybridClock initialized: now 1790708773546366 us; error 43 us; skew 500 ppm
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:13.548992 2789 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:06:13.549526 2789 webserver.cc:460] Webserver started at http://127.26.78.193:37523/ using document root <none> and password file <none>
I20260929 19:06:13.549769 2789 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:13.549860 2789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:13.550002 2789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:13.551065 2789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "44a94906a7e74be1b517df00eecf12a1"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "70fd8b936b57ffcdaa352f965da1a67d"
server_key_iv: "5308369890e97b843f439d1587b47c8f"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.551455 2789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "44a94906a7e74be1b517df00eecf12a1"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "70fd8b936b57ffcdaa352f965da1a67d"
server_key_iv: "5308369890e97b843f439d1587b47c8f"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.552887 2789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:06:13.553654 2804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.553915 2789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:13.554028 2789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "44a94906a7e74be1b517df00eecf12a1"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "70fd8b936b57ffcdaa352f965da1a67d"
server_key_iv: "5308369890e97b843f439d1587b47c8f"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.554114 2789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:13.571846 2789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:13.574080 2789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:13.574252 2789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:13.574527 2789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:13.574874 2789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:13.574934 2789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.574975 2789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:13.575003 2789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.581284 2789 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:37417
I20260929 19:06:13.581326 2917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:37417 every 8 connection(s)
I20260929 19:06:13.581666 2789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:06:13.585156 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 2789
I20260929 19:06:13.585282 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:06:13.585498 26939 external_mini_cluster.cc:1428] Setting key 5ad7a1b9417dd5e7801f05bc778b8c57
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:13.592173 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:13.582415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:59247 (local address 127.26.78.254:33151)
0929 19:06:13.582463 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:13.582466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:13.582708 (+ 242us) server_negotiation.cc:407] Connection header received
0929 19:06:13.583259 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:13.583261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:13.583273 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:13.583337 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:13.584371 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.584876 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.586151 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.586469 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.586541 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:13.588925 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:13.588926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:13.588927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:13.588934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:13.590589 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.591092 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.591093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.591094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.591159 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.591448 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.591449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.591450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.591572 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:13.591658 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:13.591976 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 19:06:13.592082 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:06:13.592401 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:13.582368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:59247)
0929 19:06:13.582537 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:13.582580 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:13.583162 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:13.583420 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:13.583424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:13.584055 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:13.584261 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:13.584265 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.584962 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.584963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:13.586054 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:13.586057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.586553 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.586553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:13.586659 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:13.587203 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:13.587212 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:13.588824 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:13.590758 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:13.590766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:13.590768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:13.591002 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:13.591243 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:13.591243 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:13.591244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:13.591360 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:13.591640 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:13.591647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:13.591826 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 19:06:13.591943 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 19:06:13.592112 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260929 19:06:13.593169 2918 heartbeater.cc:347] Connected to a master server at 127.26.78.254:33151
I20260929 19:06:13.593281 2918 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:13.593469 2918 heartbeater.cc:511] Master 127.26.78.254:33151 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
I20260929 19:06:13.594069 2726 ts_manager.cc:194] Registered new tserver with Master: 44a94906a7e74be1b517df00eecf12a1 (127.26.78.193:37417)
I20260929 19:06:13.594767 2726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:59247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:06:13.613029 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:33151
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:13.693637 2923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:13.693923 2923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:13.693984 2923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:13.695212 2923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:13.695293 2923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:13.695335 2923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:06:13.696851 2923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:33151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:13.697137 2923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:13.697386 2923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:13.702608 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:13.702615 2928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:13.702608 2931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:13.702754 2923 server_base.cc:1034] running on GCE node
I20260929 19:06:13.703239 2923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:13.703496 2923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:13.704638 2923 hybrid_clock.cc:648] HybridClock initialized: now 1790708773704622 us; error 42 us; skew 500 ppm
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:13.707309 2923 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:06:13.707811 2923 webserver.cc:460] Webserver started at http://127.26.78.194:43699/ using document root <none> and password file <none>
I20260929 19:06:13.708050 2923 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:13.708118 2923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:13.708256 2923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:13.709333 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "e99a154bb58844a2b53036ceaae3d196"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "a422b62cba5aed6edeb4476bd7291706"
server_key_iv: "8a5c909a69b682319522e81b117c06bb"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.709697 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "e99a154bb58844a2b53036ceaae3d196"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "a422b62cba5aed6edeb4476bd7291706"
server_key_iv: "8a5c909a69b682319522e81b117c06bb"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.711041 2923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:13.711786 2938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.712013 2923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:06:13.712111 2923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "e99a154bb58844a2b53036ceaae3d196"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "a422b62cba5aed6edeb4476bd7291706"
server_key_iv: "8a5c909a69b682319522e81b117c06bb"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.712198 2923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:13.720067 2923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:13.721803 2923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:13.721997 2923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:13.722177 2923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:13.722431 2923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:13.722462 2923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.722484 2923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:13.722497 2923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.728422 2923 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39973
I20260929 19:06:13.728467 3051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39973 every 8 connection(s)
I20260929 19:06:13.728773 2923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:13.737958 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 2923
I20260929 19:06:13.738106 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:06:13.738356 26939 external_mini_cluster.cc:1428] Setting key 8e089c069070c744f49e6d41fd033d2c
I20260929 19:06:13.738819 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:13.729530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:33035 (local address 127.26.78.254:33151)
0929 19:06:13.729622 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:13.729624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:13.729722 (+ 98us) server_negotiation.cc:407] Connection header received
0929 19:06:13.730297 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:13.730298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:13.730308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:13.730371 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:13.731407 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.732058 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.733144 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.733332 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.733391 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:13.735784 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:13.735785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:13.735785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:13.735792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:13.737126 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.737525 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.737526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.737526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.737583 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.738011 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.738012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.738013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.738202 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:13.738268 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:13.738625 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 19:06:13.738734 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 19:06:13.739013 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:13.729440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:33035)
0929 19:06:13.729593 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:13.729645 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:13.730202 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:13.730425 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:13.730428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:13.731083 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:13.731283 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:13.731287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.732159 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.732160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:13.733051 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:13.733054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.733410 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.733410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:13.733511 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:13.734215 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:13.734223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:13.735651 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:13.737207 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:13.737213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:13.737215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:13.737454 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:13.737680 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:13.737681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:13.737682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:13.737937 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:13.738268 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:13.738275 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:13.738464 (+ 189us) client_negotiation.cc:769] Sending connection context
0929 19:06:13.738581 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 19:06:13.738759 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260929 19:06:13.739471 3052 heartbeater.cc:347] Connected to a master server at 127.26.78.254:33151
I20260929 19:06:13.739573 3052 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:13.739778 3052 heartbeater.cc:511] Master 127.26.78.254:33151 requested a full tablet report, sending...
I20260929 19:06:13.740160 2726 ts_manager.cc:194] Registered new tserver with Master: e99a154bb58844a2b53036ceaae3d196 (127.26.78.194:39973)
I20260929 19:06:13.740538 2726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:33035
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:06:13.765481 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:33151
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:13.849040 3057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:13.849309 3057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:13.849354 3057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:13.850731 3057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:13.850807 3057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:13.850854 3057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:06:13.852402 3057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:33151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:13.852777 3057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:13.853021 3057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:13.858456 3062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:13.858461 3063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:13.858536 3057 server_base.cc:1034] running on GCE node
W20260929 19:06:13.858461 3065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:13.858873 3057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:13.859122 3057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:13.860285 3057 hybrid_clock.cc:648] HybridClock initialized: now 1790708773860275 us; error 39 us; skew 500 ppm
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:13.862773 3057 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:06:13.863260 3057 webserver.cc:460] Webserver started at http://127.26.78.195:37061/ using document root <none> and password file <none>
I20260929 19:06:13.863488 3057 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:13.863554 3057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:13.863704 3057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:13.864753 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "b6b3202145c4407ba427a26af1a704eb"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "ea5bf470639757baff8da77a661f2356"
server_key_iv: "48a63c0db741f3ae1af47948eba42805"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.865111 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "b6b3202145c4407ba427a26af1a704eb"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "ea5bf470639757baff8da77a661f2356"
server_key_iv: "48a63c0db741f3ae1af47948eba42805"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.866591 3057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:06:13.867347 3072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.867547 3057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:13.867645 3057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "b6b3202145c4407ba427a26af1a704eb"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "ea5bf470639757baff8da77a661f2356"
server_key_iv: "48a63c0db741f3ae1af47948eba42805"
server_key_version: "encryptionkey@0"
I20260929 19:06:13.867727 3057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:13.884246 3057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:13.886281 3057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:13.886451 3057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:13.886689 3057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:13.886997 3057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:13.887051 3057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.887094 3057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:13.887121 3057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:13.893061 3057 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:40615
I20260929 19:06:13.893108 3185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:40615 every 8 connection(s)
I20260929 19:06:13.893491 3057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:06:13 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:13.900434 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3057
I20260929 19:06:13.900511 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:06:13.900691 26939 external_mini_cluster.cc:1428] Setting key c071de5a49bd7d90d5a78d504c35097c
I20260929 19:06:13.903317 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:13.894280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:40795 (local address 127.26.78.254:33151)
0929 19:06:13.894331 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:13.894333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:13.894564 (+ 231us) server_negotiation.cc:407] Connection header received
0929 19:06:13.895094 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:13.895095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:13.895108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:13.895172 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:13.896162 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.896666 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.897679 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.897936 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.898022 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:13.900118 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:13.900119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:13.900120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:13.900129 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:13.901898 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.902321 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.902322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.902323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.902383 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:13.902639 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:13.902640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:13.902640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:13.902772 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:13.902863 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:13.903147 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 19:06:13.903228 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:06:13.903534 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:13.894252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:40795)
0929 19:06:13.894431 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:13.894468 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:13.894984 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:13.895253 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:13.895258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:13.895846 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:13.896040 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:13.896044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.896759 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.896760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:13.897577 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:13.897579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:13.898034 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:13.898035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:13.898167 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:13.898640 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:13.898649 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:13.900016 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:13.901994 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:13.902001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:13.902003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:13.902232 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:13.902453 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:13.902454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:13.902454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:13.902569 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:13.902856 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:13.902863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:13.903009 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 19:06:13.903115 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 19:06:13.903281 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260929 19:06:13.903998 3186 heartbeater.cc:347] Connected to a master server at 127.26.78.254:33151
I20260929 19:06:13.904103 3186 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:13.904281 3186 heartbeater.cc:511] Master 127.26.78.254:33151 requested a full tablet report, sending...
I20260929 19:06:13.904649 2726 ts_manager.cc:194] Registered new tserver with Master: b6b3202145c4407ba427a26af1a704eb (127.26.78.195:40615)
I20260929 19:06:13.904994 2726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:40795
I20260929 19:06:13.912482 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab.
Entry for principal oryx/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab.
WARNING: no policy specified for oryx/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab.
Entry for principal oryx/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab.
WARNING: no policy specified for oryx/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab.
Entry for principal oryx/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab.
I20260929 19:06:13.981308 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 2789
I20260929 19:06:13.985899 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 2923
I20260929 19:06:13.990051 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3057
I20260929 19:06:13.994227 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 2694
I20260929 19:06:13.999545 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33151
--webserver_interface=127.26.78.254
--webserver_port=35105
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:33151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:14.080583 3195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:14.080792 3195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:14.080816 3195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:14.082123 3195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:06:14.082177 3195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:14.082193 3195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:14.082206 3195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:06:14.082218 3195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:06:14.083745 3195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:33151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:14.084028 3195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:14.084271 3195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:14.089375 3201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:14.089401 3200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:14.089504 3195 server_base.cc:1034] running on GCE node
W20260929 19:06:14.089398 3203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:14.089984 3195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:14.090286 3195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:14.091454 3195 hybrid_clock.cc:648] HybridClock initialized: now 1790708774091445 us; error 29 us; skew 500 ppm
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708774, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:14.094127 3195 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:06:14.094534 3195 webserver.cc:460] Webserver started at http://127.26.78.254:35105/ using document root <none> and password file <none>
I20260929 19:06:14.094741 3195 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:14.094791 3195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:14.096097 3195 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:14.096794 3210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.096993 3195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:14.097060 3195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "e6edcb8f80c64d41848fd6336e13cb0a"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "03eea8ebc20f6bda099b41cef1285d1f"
server_key_iv: "b6313567a3c527b2bdfbdda75d9bc9dc"
server_key_version: "encryptionkey@0"
I20260929 19:06:14.097401 3195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:14.112056 3195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:14.114380 3195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:14.114511 3195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:14.118391 3195 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:33151
I20260929 19:06:14.118454 3262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:33151 every 8 connection(s)
I20260929 19:06:14.118845 3195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:06:14.120433 3263 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:06:14.121122 3263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Bootstrap starting.
I20260929 19:06:14.124320 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3195
I20260929 19:06:14.124617 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:37417
--local_ip_for_outbound_sockets=127.26.78.193
--tserver_master_addrs=127.26.78.254:33151
--webserver_port=37523
--webserver_interface=127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:06:14.129645 3263 log.cc:824] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:14.131448 3263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:06:14.131748 3263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Bootstrap complete.
I20260929 19:06:14.134044 3263 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:14.134298 3263 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6edcb8f80c64d41848fd6336e13cb0a, State: Initialized, Role: FOLLOWER
I20260929 19:06:14.134517 3263 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:14.134616 3263 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:06:14.134670 3263 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:06:14.134716 3263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:06:14.135697 3263 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:14.135821 3263 leader_election.cc:302] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6edcb8f80c64d41848fd6336e13cb0a; no voters:
I20260929 19:06:14.136035 3263 leader_election.cc:288] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:06:14.136077 3267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:06:14.136250 3267 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [term 2 LEADER]: Becoming Leader. State: Replica: e6edcb8f80c64d41848fd6336e13cb0a, State: Running, Role: LEADER
I20260929 19:06:14.136333 3263 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:06:14.136430 3267 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } }
I20260929 19:06:14.136801 3268 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } } }
I20260929 19:06:14.136910 3268 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:14.137127 3269 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6edcb8f80c64d41848fd6336e13cb0a. Latest consensus state: current_term: 2 leader_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6edcb8f80c64d41848fd6336e13cb0a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33151 } } }
I20260929 19:06:14.137197 3269 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:14.137303 3276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:06:14.137936 3276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:06:14.138116 3276 catalog_manager.cc:1259] Loaded cluster ID: 5c2a85358e7c4184ba0cd60a48f68658
I20260929 19:06:14.138319 3276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:06:14.139453 3276 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:06:14.139607 3276 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e6edcb8f80c64d41848fd6336e13cb0a: Loaded TSK: 0
I20260929 19:06:14.139806 3276 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:06:14.209666 3265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:14.209939 3265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:14.209967 3265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:14.211158 3265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:14.211200 3265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:14.211228 3265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:06:14.212698 3265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.193.keytab
--principal=oryx/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:37417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=37523
--tserver_master_addrs=127.26.78.254:33151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:14.212952 3265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:14.213198 3265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:14.218279 3292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:14.218297 3290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:14.218328 3289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:14.218837 3265 server_base.cc:1034] running on GCE node
I20260929 19:06:14.219036 3265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:14.219300 3265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:14.220468 3265 hybrid_clock.cc:648] HybridClock initialized: now 1790708774220437 us; error 53 us; skew 500 ppm
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708774, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:14.223060 3265 init.cc:377] Logged in from keytab as oryx/127.26.78.193@KRBTEST.COM (short username oryx)
I20260929 19:06:14.223488 3265 webserver.cc:460] Webserver started at http://127.26.78.193:37523/ using document root <none> and password file <none>
I20260929 19:06:14.223731 3265 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:14.223814 3265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:14.225212 3265 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:14.225921 3299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.226101 3265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:14.226202 3265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "44a94906a7e74be1b517df00eecf12a1"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "70fd8b936b57ffcdaa352f965da1a67d"
server_key_iv: "5308369890e97b843f439d1587b47c8f"
server_key_version: "encryptionkey@0"
I20260929 19:06:14.226562 3265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:14.235296 3265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:14.237964 3265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:14.238131 3265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:14.238375 3265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:14.238730 3265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:14.238781 3265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.238813 3265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:14.238844 3265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.246250 3265 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:37417
I20260929 19:06:14.246297 3412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:37417 every 8 connection(s)
I20260929 19:06:14.246717 3265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:06:14.250064 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3265
I20260929 19:06:14.250373 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:39973
--local_ip_for_outbound_sockets=127.26.78.194
--tserver_master_addrs=127.26.78.254:33151
--webserver_port=43699
--webserver_interface=127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.257982 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.247553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:39333)
0929 19:06:14.247772 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.247821 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.248461 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.248960 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.248964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.249738 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.250087 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.250094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.251764 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.251766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.253080 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.253083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.253999 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.254000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.254139 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.254773 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.254783 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.257663 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
W20260929 19:06:14.258069 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.247688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:39333 (local address 127.26.78.254:33151)
0929 19:06:14.247888 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.247895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.247922 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:06:14.248658 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.248664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.248708 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.248868 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.250234 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.251652 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.253184 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.253767 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.253906 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.257868 (+ 3962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:39333: BlockingRecv error: recv got EOF from 127.26.78.193:39333 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
W20260929 19:06:14.258644 3413 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:33151 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.78.254:33151: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.264514 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.258893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:35239)
0929 19:06:14.258938 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.258948 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.258988 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.259135 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.259136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.259313 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.259375 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.259378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.260318 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.260319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.261222 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.261224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.261719 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.261720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.261792 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.262303 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.262309 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.264449 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 19:06:14.264778 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.258943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:35239 (local address 127.26.78.254:33151)
0929 19:06:14.259004 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.259006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.259014 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:14.259039 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.259040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.259052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.259116 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.259456 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.260229 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.261326 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.261632 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.261716 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.264722 (+ 3006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:35239: BlockingRecv error: recv got EOF from 127.26.78.193:35239 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.270493 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.265311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:51071)
0929 19:06:14.265356 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.265366 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.265403 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.265533 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.265535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.265675 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.265708 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.265709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.266582 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.266583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.267448 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.267450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.267915 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.267916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.267974 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.268417 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.268422 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.270423 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 19:06:14.270572 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.265361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:51071 (local address 127.26.78.254:33151)
0929 19:06:14.265398 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.265400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.265409 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:14.265451 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.265452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.265462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.265519 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.265774 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.266505 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.267535 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.267835 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.267899 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.270523 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:51071: BlockingRecv error: recv got EOF from 127.26.78.193:51071 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 19:06:14.270843 3413 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:06:14.337442 3417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:14.337680 3417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:14.337720 3417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:14.339030 3417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:14.339099 3417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:14.339145 3417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:06:14.340562 3417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.194.keytab
--principal=oryx/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:39973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=43699
--tserver_master_addrs=127.26.78.254:33151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:14.340837 3417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:14.341075 3417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:14.346087 3423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:14.346103 3422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:14.346113 3425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:14.346386 3417 server_base.cc:1034] running on GCE node
I20260929 19:06:14.346624 3417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:14.346881 3417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:14.348040 3417 hybrid_clock.cc:648] HybridClock initialized: now 1790708774348025 us; error 27 us; skew 500 ppm
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708774, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:14.350661 3417 init.cc:377] Logged in from keytab as oryx/127.26.78.194@KRBTEST.COM (short username oryx)
I20260929 19:06:14.351074 3417 webserver.cc:460] Webserver started at http://127.26.78.194:43699/ using document root <none> and password file <none>
I20260929 19:06:14.351284 3417 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:14.351346 3417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:14.352643 3417 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:14.353128 3432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.353298 3417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.353382 3417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "e99a154bb58844a2b53036ceaae3d196"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "a422b62cba5aed6edeb4476bd7291706"
server_key_iv: "8a5c909a69b682319522e81b117c06bb"
server_key_version: "encryptionkey@0"
I20260929 19:06:14.353688 3417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:14.368588 3417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:14.370642 3417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:14.370755 3417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:14.370950 3417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:14.371276 3417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:14.371310 3417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.371333 3417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:14.371346 3417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.377084 3417 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39973
I20260929 19:06:14.377146 3546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39973 every 8 connection(s)
I20260929 19:06:14.377481 3417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.384693 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.378202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:40021)
0929 19:06:14.378365 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.378406 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.378877 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.379094 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.379097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.379671 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.379867 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.379872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.380501 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.380501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.381443 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.381446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.381777 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.381778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.381900 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.382443 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.382451 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.384548 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
W20260929 19:06:14.384709 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.378227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:40021 (local address 127.26.78.254:33151)
0929 19:06:14.378300 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.378302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.378497 (+ 195us) server_negotiation.cc:407] Connection header received
0929 19:06:14.378958 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.378959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.378973 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.379037 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.379947 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.380420 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.381518 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.381702 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.381759 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.384658 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:40021: BlockingRecv error: recv got EOF from 127.26.78.194:40021 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 19:06:14.385035 3547 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:33151 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.78.254:33151: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
I20260929 19:06:14.385913 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3417
I20260929 19:06:14.386204 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:40615
--local_ip_for_outbound_sockets=127.26.78.195
--tserver_master_addrs=127.26.78.254:33151
--webserver_port=37061
--webserver_interface=127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.389767 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.385258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:59945)
0929 19:06:14.385305 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.385315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.385379 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.385495 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.385496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.385618 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.385659 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.385661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.386423 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.386424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.386918 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.386919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.387294 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.387295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.387370 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.387820 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.387825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.389666 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 19:06:14.389796 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.385332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:59945 (local address 127.26.78.254:33151)
0929 19:06:14.385394 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.385396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.385405 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:14.385439 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.385439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.385448 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.385494 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.385749 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.386317 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.387006 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.387199 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.387294 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.389752 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:59945: BlockingRecv error: recv got EOF from 127.26.78.194:59945 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.394680 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.390046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:59555)
0929 19:06:14.390176 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.390187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.390237 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.390377 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.390378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.390528 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.390566 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.390568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.391163 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.391164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.392063 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.392066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.392382 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.392383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.392447 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.392821 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.392834 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.394630 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260929 19:06:14.394847 3547 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:06:14.395009 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.390096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:59555 (local address 127.26.78.254:33151)
0929 19:06:14.390128 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.390130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.390243 (+ 113us) server_negotiation.cc:407] Connection header received
0929 19:06:14.390298 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.390299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.390307 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.390354 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.390635 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.391086 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.392142 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.392312 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.392360 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.394932 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:59555: BlockingRecv error: recv got EOF from 127.26.78.194:59555 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 19:06:14.468189 3550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:14.468408 3550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:14.468431 3550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:14.469535 3550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:14.469573 3550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:14.469597 3550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:06:14.470960 3550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/krb5kdc/oryx_127.26.78.195.keytab
--principal=oryx/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:40615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=37061
--tserver_master_addrs=127.26.78.254:33151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:14.471150 3550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:14.471318 3550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:14.476496 3556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:14.476523 3557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:14.476675 3550 server_base.cc:1034] running on GCE node
W20260929 19:06:14.476747 3559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:14.477000 3550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:14.477263 3550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:14.478439 3550 hybrid_clock.cc:648] HybridClock initialized: now 1790708774478418 us; error 39 us; skew 500 ppm
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708774, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:14.480929 3550 init.cc:377] Logged in from keytab as oryx/127.26.78.195@KRBTEST.COM (short username oryx)
I20260929 19:06:14.481459 3550 webserver.cc:460] Webserver started at http://127.26.78.195:37061/ using document root <none> and password file <none>
I20260929 19:06:14.481652 3550 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:14.481706 3550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:14.483273 3550 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:14.484011 3566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.484200 3550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:14.484313 3550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "b6b3202145c4407ba427a26af1a704eb"
format_stamp: "Formatted at 2026-09-29 19:06:13 on dist-test-slave-xz51"
server_key: "ea5bf470639757baff8da77a661f2356"
server_key_iv: "48a63c0db741f3ae1af47948eba42805"
server_key_version: "encryptionkey@0"
I20260929 19:06:14.484726 3550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:14.509249 3550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:14.511332 3550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:14.511508 3550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:14.511742 3550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:14.512085 3550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:14.512142 3550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.512166 3550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:14.512207 3550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:14.517972 3550 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:40615
I20260929 19:06:14.518041 3679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:40615 every 8 connection(s)
I20260929 19:06:14.518447 3550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:06:14.521740 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3550
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 19:06:14.526607 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.519337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:55301 (local address 127.26.78.254:33151)
0929 19:06:14.519385 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.519386 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.519649 (+ 263us) server_negotiation.cc:407] Connection header received
0929 19:06:14.520191 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.520192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.520205 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.520262 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.521243 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.521733 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.523132 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.523328 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.523434 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.526547 (+ 3113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:55301: BlockingRecv error: recv got EOF from 127.26.78.195:55301 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:06:14.526695 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.519341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:55301)
0929 19:06:14.519517 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.519555 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.520082 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.520357 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.520361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.520934 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.521128 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.521132 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.521845 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.521846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.523029 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.523032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.523443 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.523444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.523580 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.524123 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.524131 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.526485 (+ 2354us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
W20260929 19:06:14.526952 3680 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:33151 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.78.254:33151: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
I20260929 19:06:14.529553 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.522310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.26.78.254:33151)
0929 19:06:14.522457 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.522459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.522470 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:06:14.522527 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.522527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.522540 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.522603 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.523185 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.523668 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.524427 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.524645 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.524712 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.526079 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:14.526088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:14.526088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:14.526098 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:14.528147 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:14.528476 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:14.528479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:14.528480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:14.528643 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:14.528863 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:14.528864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:14.528865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:14.529100 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:14.529262 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:14.529327 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 19:06:14.529439 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.531886 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.527305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:56197)
0929 19:06:14.527385 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.527395 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.527447 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.527569 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.527569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.527676 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.527720 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.527721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.528581 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.528582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.529076 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.529077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.529492 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.529493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.529557 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.530044 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.530049 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.531823 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260929 19:06:14.531944 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.527368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:56197 (local address 127.26.78.254:33151)
0929 19:06:14.527404 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.527406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.527453 (+ 47us) server_negotiation.cc:407] Connection header received
0929 19:06:14.527481 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.527482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.527492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.527561 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.527789 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.528462 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.529149 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.529427 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.529492 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.531906 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:56197: BlockingRecv error: recv got EOF from 127.26.78.195:56197 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:14 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:14.536095 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:14.532218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:34751)
0929 19:06:14.532271 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:14.532283 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:14.532327 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:14.532425 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:14.532426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:14.532524 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:14.532549 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.532550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.533117 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.533118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.533592 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:14.533594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.534061 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.534062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:14.534109 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.534566 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:14.534570 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:14.536047 (+ 1477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 19:06:14.536167 3416 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:14.532286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:34751 (local address 127.26.78.254:33151)
0929 19:06:14.532338 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:14.532340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:14.532354 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:06:14.532372 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:14.532373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:14.532380 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:14.532423 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:14.532606 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.533030 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.533722 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:14.533957 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:14.534013 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:14.536128 (+ 2115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:34751: BlockingRecv error: recv got EOF from 127.26.78.195:34751 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 19:06:14.536298 3680 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 19:06:15 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:15 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:15.275992 3685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:15.271292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:58107)
0929 19:06:15.271457 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:15.271477 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:15.271530 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:15.271681 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:15.271682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:15.271855 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:15.271906 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:15.271908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.272592 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.272593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:15.273139 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:15.273141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.273554 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.273554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:15.273633 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:15.274160 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:15.274165 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:15.275923 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":118,"threads_started":1}
W20260929 19:06:15.276129 3686 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:15.271369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:58107 (local address 127.26.78.254:33151)
0929 19:06:15.271513 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:15.271515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:15.271526 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:06:15.271595 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:15.271596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:15.271606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:15.271659 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:15.271990 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.272521 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.273215 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.273482 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.273554 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:15.276017 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:58107: BlockingRecv error: recv got EOF from 127.26.78.193:58107 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
Sep 29 19:06:15 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:15 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:15.400068 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:15.395299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:50809)
0929 19:06:15.395449 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:15.395459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:15.395518 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:15.395685 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:15.395687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:15.395853 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:15.395903 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:15.395905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.396536 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.396537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:15.397096 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:15.397098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.397437 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.397438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:15.397478 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:15.398031 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:15.398036 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:15.400003 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
W20260929 19:06:15.400223 3686 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:15.395372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:50809 (local address 127.26.78.254:33151)
0929 19:06:15.395442 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:15.395448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:15.395536 (+ 88us) server_negotiation.cc:407] Connection header received
0929 19:06:15.395562 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:15.395562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:15.395574 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:15.395628 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:15.395974 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.396460 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.397178 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.397371 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.397427 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:15.400138 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:50809: BlockingRecv error: recv got EOF from 127.26.78.194:50809 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 29 19:06:15 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:15 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:15.541329 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:15.536658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:34067)
0929 19:06:15.536829 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:15.536839 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:15.536892 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:15.537015 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:15.537015 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:15.537252 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:15.537301 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:15.537303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.538118 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.538119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:15.538680 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:15.538682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.539024 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.539024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:15.539066 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:15.539507 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:15.539511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:15.541247 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
W20260929 19:06:15.541455 3686 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:15.536725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:34067 (local address 127.26.78.254:33151)
0929 19:06:15.536782 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:15.536784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:15.536891 (+ 107us) server_negotiation.cc:407] Connection header received
0929 19:06:15.536934 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:15.536935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:15.536947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:15.536996 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:15.537411 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.537965 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.538760 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:15.538958 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:15.539008 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:15.541351 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:34067: BlockingRecv error: recv got EOF from 127.26.78.195:34067 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 19:06:16 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:16 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:16.280823 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:16.276415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:57529)
0929 19:06:16.276590 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:16.276601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:16.276664 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:16.276798 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:16.276799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:16.276975 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:16.277023 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:16.277024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.277655 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.277656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:16.278226 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:16.278228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.278563 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.278564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:16.278602 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:16.279113 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:16.279119 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:16.280757 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
W20260929 19:06:16.280977 3690 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:16.276490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:57529 (local address 127.26.78.254:33151)
0929 19:06:16.276594 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:16.276596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:16.276661 (+ 65us) server_negotiation.cc:407] Connection header received
0929 19:06:16.276707 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:16.276707 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:16.276718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:16.276775 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:16.277085 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.277585 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.278294 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.278499 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.278555 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:16.280887 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:57529: BlockingRecv error: recv got EOF from 127.26.78.193:57529 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":47,"threads_started":1}
Sep 29 19:06:16 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:16 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:16.406066 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:16.400491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:44449)
0929 19:06:16.400790 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:16.400802 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:16.400897 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:16.401071 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:16.401072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:16.401273 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:16.401324 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:16.401326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.402091 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.402092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:16.402664 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:16.402666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.403069 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.403070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:16.403132 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:16.403685 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:16.403691 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:16.406007 (+ 2316us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":97,"threads_started":1}
W20260929 19:06:16.406227 3690 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:16.400600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:44449 (local address 127.26.78.254:33151)
0929 19:06:16.400697 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:16.400700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:16.400878 (+ 178us) server_negotiation.cc:407] Connection header received
0929 19:06:16.400962 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:16.400963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:16.400979 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:16.401064 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:16.401414 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.402001 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.402762 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.402972 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.403066 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:16.406154 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:44449: BlockingRecv error: recv got EOF from 127.26.78.194:44449 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 19:06:16 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:16 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:16.546545 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:16.541907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:33031)
0929 19:06:16.542066 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:16.542080 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:16.542145 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:16.542274 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:16.542274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:16.542441 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:16.542489 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:16.542490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.543114 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.543115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:16.543656 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:16.543658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.543996 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.543996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:16.544035 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:16.544483 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:16.544488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:16.546459 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
W20260929 19:06:16.546670 3690 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:16.542056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:33031 (local address 127.26.78.254:33151)
0929 19:06:16.542122 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:16.542124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:16.542149 (+ 25us) server_negotiation.cc:407] Connection header received
0929 19:06:16.542197 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:16.542197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:16.542210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:16.542252 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:16.542552 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.543048 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.543732 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:16.543933 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:16.543988 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:16.546583 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:33031: BlockingRecv error: recv got EOF from 127.26.78.195:33031 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 19:06:17 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:17 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:17.285962 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:17.281269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:52297)
0929 19:06:17.281443 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:17.281454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:17.281512 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:17.281644 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:17.281645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:17.281882 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:17.281934 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:17.281936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.282618 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.282620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:17.283164 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:17.283166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.283535 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.283536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:17.283583 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:17.284085 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:17.284089 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:17.285888 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
W20260929 19:06:17.286070 3694 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:17.281375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:52297 (local address 127.26.78.254:33151)
0929 19:06:17.281515 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:17.281517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:17.281545 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:06:17.281566 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:17.281567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:17.281578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:17.281626 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:17.282008 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.282519 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.283240 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.283446 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.283497 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:17.285967 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:52297: BlockingRecv error: recv got EOF from 127.26.78.193:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
Sep 29 19:06:17 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:17 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:17.411263 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:17.406505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:50511)
0929 19:06:17.406680 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:17.406694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:17.406816 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:17.406968 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:17.406969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:17.407151 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:17.407195 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:17.407197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.407880 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.407881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:17.408422 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:17.408423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.408808 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.408809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:17.408876 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:17.409338 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:17.409342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:17.411146 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
W20260929 19:06:17.411303 3694 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:17.406609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:50511 (local address 127.26.78.254:33151)
0929 19:06:17.406684 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:17.406687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:17.406794 (+ 107us) server_negotiation.cc:407] Connection header received
0929 19:06:17.406895 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:17.406896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:17.406909 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:17.406961 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:17.407291 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.407792 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.408511 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.408732 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.408811 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:17.411229 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:50511: BlockingRecv error: recv got EOF from 127.26.78.194:50511 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 19:06:17 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:17 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:17.551370 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:17.547044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:60801)
0929 19:06:17.547227 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:17.547240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:17.547315 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:17.547488 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:17.547490 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:17.547645 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:17.547694 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:17.547696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.548313 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.548314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:17.548846 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:17.548847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.549152 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.549153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:17.549191 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:17.549604 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:17.549608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:17.551303 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
W20260929 19:06:17.551486 3694 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:17.547172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:60801 (local address 127.26.78.254:33151)
0929 19:06:17.547242 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:17.547244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:17.547311 (+ 67us) server_negotiation.cc:407] Connection header received
0929 19:06:17.547379 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:17.547380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:17.547393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:17.547441 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:17.547761 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.548247 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.548915 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:17.549091 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:17.549140 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:17.551413 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:60801: BlockingRecv error: recv got EOF from 127.26.78.195:60801 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 19:06:18 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:18 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:18.291345 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:18.286550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:54987)
0929 19:06:18.286681 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:18.286693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:18.286758 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:18.286888 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:18.286888 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:18.287084 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:18.287143 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:18.287145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.287854 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.287854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:18.288443 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:18.288445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.288775 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.288776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:18.288816 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:18.289335 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:18.289340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:18.291263 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
W20260929 19:06:18.291486 3698 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:18.286588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:54987 (local address 127.26.78.254:33151)
0929 19:06:18.286688 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:18.286690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:18.286761 (+ 71us) server_negotiation.cc:407] Connection header received
0929 19:06:18.286801 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:18.286801 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:18.286811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:18.286871 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:18.287224 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.287756 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.288511 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.288708 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.288766 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:18.291376 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:54987: BlockingRecv error: recv got EOF from 127.26.78.193:54987 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":47,"threads_started":1}
Sep 29 19:06:18 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:18 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:18.423858 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:18.411821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:43329)
0929 19:06:18.411968 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:18.411987 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:18.412049 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:18.412177 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:18.412177 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:18.412365 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:18.412414 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:18.412416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.413045 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.413046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:18.413573 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:18.413575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.413914 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.413917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:18.413966 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:18.421974 (+ 8008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:18.421980 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:18.423794 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
W20260929 19:06:18.424033 3698 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:18.411988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:43329 (local address 127.26.78.254:33151)
0929 19:06:18.412052 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:18.412054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:18.412066 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:06:18.412098 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:18.412098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:18.412108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:18.412156 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:18.412484 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.412978 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.413646 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.413848 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.413908 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:18.423951 (+ 10043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:43329: BlockingRecv error: recv got EOF from 127.26.78.194:43329 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 19:06:18 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:18 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:18.556317 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:18.551899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:42287)
0929 19:06:18.552035 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:18.552053 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:18.552110 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:18.552242 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:18.552243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:18.552409 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:18.552458 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:18.552460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.553093 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.553094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:18.553629 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:18.553631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.553995 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.553996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:18.554041 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:18.554495 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:18.554500 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:18.556230 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
W20260929 19:06:18.556409 3698 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:18.551970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:42287 (local address 127.26.78.254:33151)
0929 19:06:18.552044 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:18.552045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:18.552095 (+ 50us) server_negotiation.cc:407] Connection header received
0929 19:06:18.552168 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:18.552168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:18.552179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:18.552220 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:18.552524 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.553024 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.553704 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:18.553928 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:18.553983 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:18.556360 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:42287: BlockingRecv error: recv got EOF from 127.26.78.195:42287 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 19:06:19 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:19 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:19.296980 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:19.291882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:43635)
0929 19:06:19.292089 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:19.292102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:19.292201 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:19.292384 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:19.292385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:19.292561 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:19.292609 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:19.292611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.293289 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.293290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:19.293871 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:19.293873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.294238 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.294239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:19.294293 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:19.294778 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:19.294783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:19.296921 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":114,"threads_started":1}
W20260929 19:06:19.297133 3701 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:19.291922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:43635 (local address 127.26.78.254:33151)
0929 19:06:19.292077 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:19.292080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:19.292179 (+ 99us) server_negotiation.cc:407] Connection header received
0929 19:06:19.292259 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:19.292260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:19.292274 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:19.292369 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:19.292691 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.293201 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.293957 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.294156 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.294240 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:19.297049 (+ 2809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:43635: BlockingRecv error: recv got EOF from 127.26.78.193:43635 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
Sep 29 19:06:19 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:19 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:19.429050 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:19.424370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:54827)
0929 19:06:19.424529 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:19.424540 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:19.424600 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:19.424724 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:19.424725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:19.424885 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:19.424933 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:19.424935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.425575 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.425576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:19.426115 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:19.426117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.426461 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.426462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:19.426501 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:19.427100 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:19.427106 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:19.428987 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
W20260929 19:06:19.429248 3701 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:19.424543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:54827 (local address 127.26.78.254:33151)
0929 19:06:19.424593 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:19.424595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:19.424612 (+ 17us) server_negotiation.cc:407] Connection header received
0929 19:06:19.424645 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:19.424645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:19.424656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:19.424704 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:19.425005 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.425505 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.426193 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.426395 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.426453 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:19.429100 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:54827: BlockingRecv error: recv got EOF from 127.26.78.194:54827 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 29 19:06:19 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:19 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:19.561605 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:19.556993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:41369)
0929 19:06:19.557109 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:19.557125 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:19.557185 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:19.557463 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:19.557465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:19.557661 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:19.557712 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:19.557714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.558389 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.558390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:19.558928 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:19.558929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.559265 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.559266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:19.559308 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:19.559775 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:19.559779 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:19.561546 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
W20260929 19:06:19.561754 3701 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:19.557059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:41369 (local address 127.26.78.254:33151)
0929 19:06:19.557111 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:19.557113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:19.557175 (+ 62us) server_negotiation.cc:407] Connection header received
0929 19:06:19.557292 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:19.557293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:19.557308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:19.557409 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:19.557780 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.558311 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.559011 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:19.559199 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:19.559253 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:19.561674 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:41369: BlockingRecv error: recv got EOF from 127.26.78.195:41369 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 29 19:06:20 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:20 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:20.302260 3705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:20.297511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:36573)
0929 19:06:20.297638 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:20.297649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:20.297701 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:20.297947 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:20.297949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:20.298109 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:20.298159 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:20.298161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.298808 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.298809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:20.299352 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:20.299354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.299672 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.299673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:20.299712 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:20.300214 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:20.300219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:20.302198 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
W20260929 19:06:20.302492 3706 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:20.297607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:36573 (local address 127.26.78.254:33151)
0929 19:06:20.297758 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:20.297765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:20.297774 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:20.297800 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:20.297801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:20.297814 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:20.297897 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:20.298231 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.298741 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.299422 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.299610 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.299659 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:20.302347 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:36573: BlockingRecv error: recv got EOF from 127.26.78.193:36573 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
Sep 29 19:06:20 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:20 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:20.434130 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:20.429578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:41971)
0929 19:06:20.429711 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:20.429722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:20.429793 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:20.429993 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:20.429994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:20.430153 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:20.430202 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:20.430204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.430865 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.430866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:20.431393 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:20.431394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.431708 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.431709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:20.431749 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:20.432206 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:20.432211 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:20.434028 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
W20260929 19:06:20.434180 3706 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:20.429644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:41971 (local address 127.26.78.254:33151)
0929 19:06:20.429719 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:20.429721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:20.429762 (+ 41us) server_negotiation.cc:407] Connection header received
0929 19:06:20.429911 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:20.429912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:20.429922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:20.429975 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:20.430270 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.430783 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.431467 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.431648 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.431698 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:20.434115 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:41971: BlockingRecv error: recv got EOF from 127.26.78.194:41971 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 19:06:20 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:20 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:20.566980 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:20.562104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:43009)
0929 19:06:20.562237 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:20.562253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:20.562329 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:20.562494 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:20.562495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:20.562690 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:20.562737 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:20.562739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.563490 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.563491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:20.564019 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:20.564021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.564397 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.564398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:20.564463 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:20.564981 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:20.564986 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:20.566889 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
W20260929 19:06:20.567090 3706 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:20.562228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:43009 (local address 127.26.78.254:33151)
0929 19:06:20.562305 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:20.562307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:20.562317 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:06:20.562401 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:20.562402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:20.562421 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:20.562487 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:20.562870 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.563395 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.564114 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:20.564309 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:20.564387 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:20.567024 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:43009: BlockingRecv error: recv got EOF from 127.26.78.195:43009 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 19:06:21 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:21 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:21.307492 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:21.302771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:38363)
0929 19:06:21.302922 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:21.302933 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:21.302983 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:21.303141 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:21.303141 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:21.303303 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:21.303358 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:21.303359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.304009 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.304011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:21.304606 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:21.304608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.305043 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.305044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:21.305112 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:21.305554 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:21.305558 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:21.307393 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
W20260929 19:06:21.307557 3710 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:21.302881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:38363 (local address 127.26.78.254:33151)
0929 19:06:21.303009 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:21.303011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:21.303020 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:21.303044 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:21.303045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:21.303055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:21.303126 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:21.303423 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.303936 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.304703 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.304959 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.305041 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:21.307482 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:38363: BlockingRecv error: recv got EOF from 127.26.78.193:38363 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
Sep 29 19:06:21 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:21 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:21.439473 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:21.434671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:33701)
0929 19:06:21.434819 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:21.434829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:21.434892 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:21.435037 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:21.435038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:21.435201 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:21.435251 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:21.435253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.436137 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.436138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:21.436779 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:21.436781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.437105 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.437106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:21.437149 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:21.437597 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:21.437601 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:21.439412 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260929 19:06:21.439634 3710 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:21.434732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:33701 (local address 127.26.78.254:33151)
0929 19:06:21.434814 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:21.434816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:21.434891 (+ 75us) server_negotiation.cc:407] Connection header received
0929 19:06:21.434942 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:21.434943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:21.434955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:21.435021 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:21.435392 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.436061 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.436852 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.437038 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.437094 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:21.439537 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:33701: BlockingRecv error: recv got EOF from 127.26.78.194:33701 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 19:06:21 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:21 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:21.572381 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:21.567432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:54061)
0929 19:06:21.567600 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:21.567612 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:21.567704 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:21.567919 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:21.567920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:21.568100 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:21.568145 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:21.568146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.568824 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.568825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:21.569361 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:21.569364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.569732 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.569733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:21.569799 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:21.570290 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:21.570294 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:21.572287 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
W20260929 19:06:21.572458 3710 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:21.567565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:54061 (local address 127.26.78.254:33151)
0929 19:06:21.567616 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:21.567618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:21.567685 (+ 67us) server_negotiation.cc:407] Connection header received
0929 19:06:21.567786 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:21.567787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:21.567800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:21.567903 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:21.568227 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.568728 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.569451 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:21.569657 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:21.569733 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:21.572388 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:54061: BlockingRecv error: recv got EOF from 127.26.78.195:54061 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 29 19:06:22 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:22 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:22.318620 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:22.313000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:49215)
0929 19:06:22.313139 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:22.313148 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:22.313196 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:22.314084 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:22.314085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:22.314242 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:22.314290 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:22.314292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.315032 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.315032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:22.315571 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:22.315573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.316086 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.316088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:22.316154 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:22.316679 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:22.316684 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:22.318520 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
W20260929 19:06:22.318679 3714 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:22.313110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:49215 (local address 127.26.78.254:33151)
0929 19:06:22.313930 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:22.313933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:22.313944 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:06:22.313973 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:22.313974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:22.313987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:22.314069 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:22.314404 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.314955 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.315656 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.316018 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.316085 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:22.318604 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:49215: BlockingRecv error: recv got EOF from 127.26.78.193:49215 (error 108)
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":75,"threads_started":1}
Sep 29 19:06:22 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:22 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:22.444978 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:22.440020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:57423)
0929 19:06:22.440190 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:22.440200 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:22.440284 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:22.440433 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:22.440433 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:22.440613 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:22.440660 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:22.440662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.441502 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.441503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:22.442200 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:22.442202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.442585 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.442586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:22.442641 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:22.443094 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:22.443098 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:22.444882 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":89,"threads_started":1}
W20260929 19:06:22.445037 3714 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:22.440077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:57423 (local address 127.26.78.254:33151)
0929 19:06:22.440147 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:22.440149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:22.440264 (+ 115us) server_negotiation.cc:407] Connection header received
0929 19:06:22.440347 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:22.440348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:22.440365 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:22.440414 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:22.440748 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.441406 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.442279 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.442482 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.442585 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:22.444974 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:57423: BlockingRecv error: recv got EOF from 127.26.78.194:57423 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 19:06:22 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:22 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:22.577487 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:22.572999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:43709)
0929 19:06:22.573187 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:22.573199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:22.573273 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:22.573419 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:22.573420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:22.573583 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:22.573633 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:22.573634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.574346 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.574347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:22.574870 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:22.574871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.575203 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.575203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:22.575242 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:22.575687 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:22.575691 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:22.577427 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":81,"threads_started":1}
W20260929 19:06:22.577602 3714 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:22.573078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:43709 (local address 127.26.78.254:33151)
0929 19:06:22.573191 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:22.573192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:22.573251 (+ 59us) server_negotiation.cc:407] Connection header received
0929 19:06:22.573331 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:22.573332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:22.573348 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:22.573391 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:22.573707 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.574271 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.574954 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:22.575135 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:22.575193 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:22.577525 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:43709: BlockingRecv error: recv got EOF from 127.26.78.195:43709 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 29 19:06:23 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:23 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:23.323887 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:23.319132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:56081)
0929 19:06:23.319295 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:23.319310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:23.319395 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:23.319546 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:23.319547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:23.319711 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:23.319762 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:23.319763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.320422 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.320423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:23.321018 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:23.321021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.321375 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.321376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:23.321422 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:23.321885 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:23.321890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:23.323822 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
W20260929 19:06:23.324023 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:23.319270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:56081 (local address 127.26.78.254:33151)
0929 19:06:23.319423 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:23.319425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:23.319433 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:23.319456 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:23.319456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:23.319467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:23.319526 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:23.319827 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.320335 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.321104 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.321301 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.321363 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:23.323935 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:56081: BlockingRecv error: recv got EOF from 127.26.78.193:56081 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
Sep 29 19:06:23 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:23 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:23.456247 3719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:23.445540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:49693)
0929 19:06:23.445732 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:23.445745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:23.445895 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:23.446044 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:23.446045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:23.446276 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:23.446375 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:23.446378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.447074 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.447076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:23.447628 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:23.447630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.448021 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.448022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:23.448084 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:23.454210 (+ 6126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:23.454215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:23.456175 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
W20260929 19:06:23.456393 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:23.445639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:49693 (local address 127.26.78.254:33151)
0929 19:06:23.445700 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:23.445702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:23.445837 (+ 135us) server_negotiation.cc:407] Connection header received
0929 19:06:23.445960 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:23.445961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:23.445974 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:23.446028 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:23.446481 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.446982 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.447726 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.447941 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.448023 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:23.456307 (+ 8284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:49693: BlockingRecv error: recv got EOF from 127.26.78.194:49693 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 29 19:06:23 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:23 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:23.582893 3720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:23.578153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:48727)
0929 19:06:23.578269 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:23.578281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:23.578396 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:23.578587 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:23.578587 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:23.578755 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:23.578803 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:23.578804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.579450 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.579450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:23.579979 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:23.579981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.580364 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.580365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:23.580405 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:23.580903 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:23.580907 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:23.582816 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
W20260929 19:06:23.582967 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:23.578201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:48727 (local address 127.26.78.254:33151)
0929 19:06:23.578274 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:23.578277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:23.578361 (+ 84us) server_negotiation.cc:407] Connection header received
0929 19:06:23.578484 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:23.578485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:23.578503 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:23.578568 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:23.578878 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.579375 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.580071 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:23.580274 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:23.580340 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:23.582912 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:48727: BlockingRecv error: recv got EOF from 127.26.78.195:48727 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 19:06:24 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:24 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:24.329633 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:24.324500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:47945)
0929 19:06:24.324675 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:24.324687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:24.324754 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:24.324914 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:24.324915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:24.325082 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:24.325131 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:24.325133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.325951 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.325952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:24.326477 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:24.326479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.327089 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.327091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:24.327149 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:24.327639 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:24.327644 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:24.329573 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
W20260929 19:06:24.329886 3722 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:24.324617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:47945 (local address 127.26.78.254:33151)
0929 19:06:24.324730 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:24.324732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:24.324741 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:24.324819 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:24.324820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:24.324831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:24.324891 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:24.325252 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.325802 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.326633 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.326937 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.327035 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:24.329786 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:47945: BlockingRecv error: recv got EOF from 127.26.78.193:47945 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
Sep 29 19:06:24 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:24 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:24.462116 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:24.456752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:44265)
0929 19:06:24.456902 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:24.456921 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:24.456995 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:24.457165 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:24.457166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:24.457338 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:24.457387 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:24.457388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.458288 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.458290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:24.458813 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:24.458814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.459333 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.459335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:24.459401 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:24.460062 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:24.460068 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:24.462051 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
W20260929 19:06:24.462199 3722 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:24.456828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:44265 (local address 127.26.78.254:33151)
0929 19:06:24.456900 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:24.456902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:24.456988 (+ 86us) server_negotiation.cc:407] Connection header received
0929 19:06:24.457067 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:24.457068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:24.457080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:24.457124 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:24.457501 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.458188 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.458947 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.459232 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.459312 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:24.462151 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:44265: BlockingRecv error: recv got EOF from 127.26.78.194:44265 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 29 19:06:24 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:24 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:24.588228 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:24.583429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:44677)
0929 19:06:24.583574 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:24.583585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:24.583658 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:24.583797 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:24.583798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:24.583978 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:24.584026 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:24.584028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.584730 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.584731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:24.585251 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:24.585253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.585661 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.585662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:24.585726 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:24.586264 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:24.586269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:24.588130 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
W20260929 19:06:24.588282 3722 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:24.583611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:44677 (local address 127.26.78.254:33151)
0929 19:06:24.583677 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:24.583679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:24.583687 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:24.583714 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:24.583715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:24.583730 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:24.583779 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:24.584117 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.584632 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.585373 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:24.585572 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:24.585659 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:24.588218 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:44677: BlockingRecv error: recv got EOF from 127.26.78.195:44677 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 29 19:06:25 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:25 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:25.334820 3725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:25.330126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:43071)
0929 19:06:25.330268 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:25.330279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:25.330343 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:25.330485 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:25.330485 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:25.330651 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:25.330700 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:25.330701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.331405 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.331406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:25.331927 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:25.331928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.332288 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.332288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:25.332333 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:25.332769 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:25.332773 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:25.334749 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
W20260929 19:06:25.334897 3726 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:25.330212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:43071 (local address 127.26.78.254:33151)
0929 19:06:25.330368 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:25.330370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:25.330378 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:25.330401 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:25.330402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:25.330413 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:25.330472 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:25.330814 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.331320 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.332011 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.332218 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.332277 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:25.334840 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:43071: BlockingRecv error: recv got EOF from 127.26.78.193:43071 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":53,"threads_started":1}
Sep 29 19:06:25 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:25 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:25.467952 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:25.462755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:48419)
0929 19:06:25.462908 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:25.462927 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:25.462996 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:25.463131 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:25.463132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:25.463303 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:25.463353 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:25.463355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.464303 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.464303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:25.464826 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:25.464827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.465207 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.465209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:25.465262 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:25.465922 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:25.465935 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:25.467876 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
W20260929 19:06:25.468089 3726 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:25.462826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:48419 (local address 127.26.78.254:33151)
0929 19:06:25.462899 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:25.462901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:25.462998 (+ 97us) server_negotiation.cc:407] Connection header received
0929 19:06:25.463047 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:25.463048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:25.463059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:25.463110 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:25.463444 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.464197 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.464915 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.465107 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.465165 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:25.468001 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:48419: BlockingRecv error: recv got EOF from 127.26.78.194:48419 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 19:06:25 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:25 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:25.593773 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:25.588896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:39101)
0929 19:06:25.589092 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:25.589106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:25.589197 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:25.589336 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:25.589337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:25.589531 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:25.589575 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:25.589577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.590354 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.590355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:25.590908 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:25.590910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.591301 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.591302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:25.591362 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:25.591843 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:25.591848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:25.593661 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
W20260929 19:06:25.593838 3726 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:25.589033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:39101 (local address 127.26.78.254:33151)
0929 19:06:25.589113 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:25.589115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:25.589192 (+ 77us) server_negotiation.cc:407] Connection header received
0929 19:06:25.589256 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:25.589257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:25.589272 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:25.589328 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:25.589662 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.590264 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.590999 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:25.591205 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:25.591293 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:25.593741 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:39101: BlockingRecv error: recv got EOF from 127.26.78.195:39101 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 19:06:26 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:26 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:26.340457 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:26.335346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:51375)
0929 19:06:26.335482 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:26.335497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:26.335571 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:26.335724 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:26.335725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:26.335963 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:26.336025 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:26.336027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.336904 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.336905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:26.337557 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:26.337559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.338008 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.338009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:26.338066 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:26.338537 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:26.338541 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:26.340359 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
W20260929 19:06:26.340512 3730 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:26.335416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:51375 (local address 127.26.78.254:33151)
0929 19:06:26.335579 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:26.335581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:26.335593 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:06:26.335620 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:26.335621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:26.335633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:26.335714 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:26.336139 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.336811 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.337641 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.337922 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.338007 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:26.340439 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:51375: BlockingRecv error: recv got EOF from 127.26.78.193:51375 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
Sep 29 19:06:26 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:26 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:26.473423 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:26.468506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:36699)
0929 19:06:26.468645 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:26.468656 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:26.468710 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:26.468879 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:26.468879 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:26.469050 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:26.469096 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:26.469097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.469859 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.469860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:26.470407 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:26.470408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.470839 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.470839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:26.470885 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:26.471434 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:26.471439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:26.473351 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
W20260929 19:06:26.473495 3730 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:26.468581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:36699 (local address 127.26.78.254:33151)
0929 19:06:26.468679 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:26.468681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:26.468699 (+ 18us) server_negotiation.cc:407] Connection header received
0929 19:06:26.468787 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:26.468788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:26.468802 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:26.468867 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:26.469201 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.469701 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.470498 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.470757 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.470817 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:26.473447 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:36699: BlockingRecv error: recv got EOF from 127.26.78.194:36699 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 19:06:26 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:26 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:26.598707 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:26.594289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:33311)
0929 19:06:26.594454 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:26.594464 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:26.594527 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:26.594650 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:26.594651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:26.594817 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:26.594871 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:26.594873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.595498 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.595499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:26.596014 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:26.596015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.596350 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.596352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:26.596403 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:26.596892 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:26.596897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:26.598648 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
W20260929 19:06:26.598816 3730 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:26.594388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:33311 (local address 127.26.78.254:33151)
0929 19:06:26.594460 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:26.594462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:26.594513 (+ 51us) server_negotiation.cc:407] Connection header received
0929 19:06:26.594588 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:26.594589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:26.594601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:26.594638 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:26.594934 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.595429 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.596086 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:26.596259 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:26.596310 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:26.598742 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:33311: BlockingRecv error: recv got EOF from 127.26.78.195:33311 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 19:06:27 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:27 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:27.352061 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:27.341001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:60499)
0929 19:06:27.341141 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:27.341151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:27.341203 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:27.341398 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:27.341398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:27.341566 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:27.341614 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:27.341616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.342384 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.342385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:27.342952 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:27.342955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.343296 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.343298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:27.343378 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:27.350089 (+ 6711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:27.350095 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:27.351991 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
W20260929 19:06:27.352219 3734 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:27.341114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:60499 (local address 127.26.78.254:33151)
0929 19:06:27.341251 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:27.341253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:27.341261 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:27.341286 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:27.341287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:27.341298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:27.341374 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:27.341698 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.342289 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.343027 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.343216 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.343270 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:27.352118 (+ 8848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:60499: BlockingRecv error: recv got EOF from 127.26.78.193:60499 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
Sep 29 19:06:27 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:27 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:27.478940 3735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:27.474081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:50293)
0929 19:06:27.474243 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:27.474257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:27.474336 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:27.474558 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:27.474559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:27.474795 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:27.474853 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:27.474856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.475531 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.475532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:27.476037 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:27.476038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.476407 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.476408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:27.476444 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:27.476963 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:27.476969 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:27.478878 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
W20260929 19:06:27.479065 3734 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:27.474270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:50293 (local address 127.26.78.254:33151)
0929 19:06:27.474319 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:27.474321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:27.474333 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:06:27.474433 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:27.474434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:27.474446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:27.474527 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:27.474946 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.475460 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.476134 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.476331 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.476400 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:27.478982 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:50293: BlockingRecv error: recv got EOF from 127.26.78.194:50293 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 29 19:06:27 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:27 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:27.604105 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:27.599305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:41441)
0929 19:06:27.599496 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:27.599507 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:27.599599 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:27.599779 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:27.599780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:27.600038 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:27.600086 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:27.600088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.600783 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.600784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:27.601382 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:27.601384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.601735 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.601736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:27.601795 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:27.602298 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:27.602303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:27.604049 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":108,"threads_started":1}
W20260929 19:06:27.604229 3734 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:27.599407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:41441 (local address 127.26.78.254:33151)
0929 19:06:27.599478 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:27.599481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:27.599580 (+ 99us) server_negotiation.cc:407] Connection header received
0929 19:06:27.599653 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:27.599654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:27.599691 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:27.599766 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:27.600164 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.600687 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.601464 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:27.601659 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:27.601736 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:27.604147 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:41441: BlockingRecv error: recv got EOF from 127.26.78.195:41441 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 29 19:06:28 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:28 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:28.357162 3737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:28.352554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:46781)
0929 19:06:28.352726 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:28.352740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:28.352805 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:28.352955 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:28.352956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:28.353121 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:28.353169 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:28.353171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.353808 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.353809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:28.354392 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:28.354394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.354717 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.354717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:28.354755 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:28.355209 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:28.355213 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:28.357104 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":71,"threads_started":1}
W20260929 19:06:28.357476 3738 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:28.352612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:46781 (local address 127.26.78.254:33151)
0929 19:06:28.352788 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:28.352790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:28.352806 (+ 16us) server_negotiation.cc:407] Connection header received
0929 19:06:28.352854 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:28.352854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:28.352866 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:28.352927 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:28.353237 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.353739 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.354466 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.354654 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.354706 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:28.357336 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:46781: BlockingRecv error: recv got EOF from 127.26.78.193:46781 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
Sep 29 19:06:28 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:28 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:28.493072 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:28.479448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:58461)
0929 19:06:28.479588 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:28.479601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:28.479651 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:28.479919 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:28.479920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:28.480087 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:28.480136 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:28.480137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.480787 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.480788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:28.481322 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:28.481323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.481656 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.481657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:28.481703 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:28.491053 (+ 9350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:28.491059 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:28.493007 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
W20260929 19:06:28.493242 3738 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:28.479584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:58461 (local address 127.26.78.254:33151)
0929 19:06:28.479774 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:28.479776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:28.479787 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:06:28.479817 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:28.479817 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:28.479830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:28.479898 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:28.480210 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.480705 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.481398 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.481586 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.481638 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:28.493161 (+ 11523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:58461: BlockingRecv error: recv got EOF from 127.26.78.194:58461 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
Sep 29 19:06:28 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:28 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.195@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:28.609912 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:28.604618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.195:39685)
0929 19:06:28.604837 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:28.604848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:28.604920 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:28.605066 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:28.605067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:28.605246 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:28.605293 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:28.605295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.606169 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.606171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:28.606756 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:28.606759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.607206 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.607207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:28.607260 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:28.607762 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:28.607767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:28.609763 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":161,"threads_started":1}
W20260929 19:06:28.609951 3738 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:28.604761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:39685 (local address 127.26.78.254:33151)
0929 19:06:28.604830 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:28.604832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:28.604908 (+ 76us) server_negotiation.cc:407] Connection header received
0929 19:06:28.604966 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:28.604967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:28.604981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:28.605043 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:28.605375 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.606056 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.606877 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:28.607090 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:28.607174 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:28.609889 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.195:39685: BlockingRecv error: recv got EOF from 127.26.78.195:39685 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 19:06:29 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:29 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.193@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:29.362571 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:29.357695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.193:33981)
0929 19:06:29.357929 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:29.357941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:29.358004 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:29.358158 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:29.358159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:29.358330 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:29.358384 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.358385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.359088 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.359090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.359686 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.359688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.360064 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.360065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.360108 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.360539 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:29.360544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:29.362461 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":69,"threads_started":1}
W20260929 19:06:29.362628 3742 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:29.357758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:33981 (local address 127.26.78.254:33151)
0929 19:06:29.357952 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:29.357954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:29.357987 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:06:29.358064 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:29.358064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:29.358075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:29.358136 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:29.358483 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.359004 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.359770 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.359992 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.360048 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.362542 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.193:33981: BlockingRecv error: recv got EOF from 127.26.78.193:33981 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":120,"threads_started":1}
Sep 29 19:06:29 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:06:29 dist-test-slave-xz51 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.78.194@KRBTEST.COM for oryx/127.26.78.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:06:29.498751 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:29.493637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33151 (local address 127.26.78.194:34869)
0929 19:06:29.493890 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:29.493901 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:29.493990 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:29.494158 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:29.494158 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:29.494351 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:29.494401 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.494423 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.495314 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.495315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.495875 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.495877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.496273 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.496274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.496341 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.496863 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:29.496868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:29.498682 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.78.254:33151: Server oryx/127.26.78.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":83,"threads_started":1}
W20260929 19:06:29.498834 3742 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:29.493744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:34869 (local address 127.26.78.254:33151)
0929 19:06:29.493815 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:29.493841 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:29.493968 (+ 127us) server_negotiation.cc:407] Connection header received
0929 19:06:29.494061 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:29.494062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:29.494082 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:29.494149 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:29.494532 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.495200 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.495971 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.496167 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.496227 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.498784 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.78.194:34869: BlockingRecv error: recv got EOF from 127.26.78.194:34869 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:06:29.526717 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3265
I20260929 19:06:29.531406 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3417
I20260929 19:06:29.535586 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3550
I20260929 19:06:29.540042 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3195
2026-09-29T19:06:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17458 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T19:06:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:06:29Z Disabled control of system clock
I20260929 19:06:29.567715 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:42623
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:42623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 19:06:29.649978 3747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:29.650194 3747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:29.650215 3747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:29.651499 3747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:06:29.651551 3747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:29.651564 3747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:29.651577 3747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:06:29.651589 3747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:06:29.653100 3747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:42623
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:42623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:29.653398 3747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:29.653728 3747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:29.659126 3755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:29.659117 3752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:29.659117 3753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:29.659190 3747 server_base.cc:1034] running on GCE node
I20260929 19:06:29.659685 3747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:29.659986 3747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:29.661161 3747 hybrid_clock.cc:648] HybridClock initialized: now 1790708789661152 us; error 42 us; skew 500 ppm
I20260929 19:06:29.662488 3747 webserver.cc:460] Webserver started at http://127.26.78.254:38239/ using document root <none> and password file <none>
I20260929 19:06:29.662750 3747 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:29.662820 3747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:29.662968 3747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:29.664052 3747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "29464775464140eab72c41c7396c8d99"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "3ede910800c65004bb7b2bfcedbd9e31"
server_key_iv: "265fbdf57c8642abe52f84ab0111e8ef"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.664419 3747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "29464775464140eab72c41c7396c8d99"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "3ede910800c65004bb7b2bfcedbd9e31"
server_key_iv: "265fbdf57c8642abe52f84ab0111e8ef"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.665726 3747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:06:29.666509 3761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:29.666761 3747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:29.666862 3747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "29464775464140eab72c41c7396c8d99"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "3ede910800c65004bb7b2bfcedbd9e31"
server_key_iv: "265fbdf57c8642abe52f84ab0111e8ef"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.666944 3747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:29.692770 3747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:29.693127 3747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:29.693298 3747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:29.697683 3747 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:42623
I20260929 19:06:29.697779 3813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:42623 every 8 connection(s)
I20260929 19:06:29.698176 3747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:06:29.698870 3814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:29.701452 3814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99: Bootstrap starting.
I20260929 19:06:29.702250 3814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:29.702514 3814 log.cc:824] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:29.702651 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3747
I20260929 19:06:29.702750 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:06:29.702979 26939 external_mini_cluster.cc:1428] Setting key 14f4bb222aec7a2e915101d6c797b41b
I20260929 19:06:29.703401 3814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99: No bootstrap required, opened a new log
I20260929 19:06:29.705168 3814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29464775464140eab72c41c7396c8d99" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42623 } }
I20260929 19:06:29.705348 3814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:29.705386 3814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29464775464140eab72c41c7396c8d99, State: Initialized, Role: FOLLOWER
I20260929 19:06:29.705575 3814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29464775464140eab72c41c7396c8d99" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42623 } }
I20260929 19:06:29.705679 3814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:06:29.705704 3814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:06:29.705780 3814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:29.706619 3814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29464775464140eab72c41c7396c8d99" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42623 } }
I20260929 19:06:29.706719 3814 leader_election.cc:302] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29464775464140eab72c41c7396c8d99; no voters:
I20260929 19:06:29.706943 3814 leader_election.cc:288] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:06:29.706967 3819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:29.707084 3819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [term 1 LEADER]: Becoming Leader. State: Replica: 29464775464140eab72c41c7396c8d99, State: Running, Role: LEADER
I20260929 19:06:29.707226 3814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:06:29.707242 3819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29464775464140eab72c41c7396c8d99" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42623 } }
I20260929 19:06:29.707688 3820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29464775464140eab72c41c7396c8d99. Latest consensus state: current_term: 1 leader_uuid: "29464775464140eab72c41c7396c8d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29464775464140eab72c41c7396c8d99" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42623 } } }
I20260929 19:06:29.707785 3820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:29.708014 3821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29464775464140eab72c41c7396c8d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29464775464140eab72c41c7396c8d99" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 42623 } } }
I20260929 19:06:29.708391 3821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99 [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:29.708329 3826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:06:29.709100 3826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:06:29.710202 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:29.703536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.26.78.254:42623)
0929 19:06:29.703740 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:29.703748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:29.703774 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:06:29.705149 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:29.705157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:29.705222 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:29.705390 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:29.705623 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.706725 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.707725 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.708354 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.708464 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.709399 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:29.709403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:29.709404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:29.709608 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:29.709666 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:29.709681 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:29.709758 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:29.709899 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 19:06:29.709987 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260929 19:06:29.711019 3826 catalog_manager.cc:1347] Generated new cluster ID: d4ff307a13f649de9f4a39b0f751f79f
I20260929 19:06:29.711102 3826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:06:29.722620 3826 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:06:29.723177 3826 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:06:29.729696 3826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29464775464140eab72c41c7396c8d99: Generated new TSK 0
I20260929 19:06:29.729925 3826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:06:29.737666 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:42623
--builtin_ntp_servers=127.26.78.212:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 19:06:29.823693 3838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:29.823907 3838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:29.823930 3838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:29.825099 3838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:29.825139 3838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:29.825165 3838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:06:29.826572 3838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:42623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:29.826817 3838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:29.827137 3838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:29.832230 3846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:29.832242 3844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:29.832242 3843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:29.832846 3838 server_base.cc:1034] running on GCE node
I20260929 19:06:29.833093 3838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:29.833385 3838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:29.834554 3838 hybrid_clock.cc:648] HybridClock initialized: now 1790708789834538 us; error 33 us; skew 500 ppm
I20260929 19:06:29.835978 3838 webserver.cc:460] Webserver started at http://127.26.78.193:40777/ using document root <none> and password file <none>
I20260929 19:06:29.836233 3838 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:29.836280 3838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:29.836438 3838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:29.837615 3838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "7aaef2104f7441859463bcef25eb240a"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "dad6157205ff5bdc9b0e1ee870178712"
server_key_iv: "8052162daf2e5137720b47e7b525bc2a"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.838007 3838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "7aaef2104f7441859463bcef25eb240a"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "dad6157205ff5bdc9b0e1ee870178712"
server_key_iv: "8052162daf2e5137720b47e7b525bc2a"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.839386 3838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:29.840366 3852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:29.840520 3838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:29.840617 3838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "7aaef2104f7441859463bcef25eb240a"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "dad6157205ff5bdc9b0e1ee870178712"
server_key_iv: "8052162daf2e5137720b47e7b525bc2a"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.840689 3838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:29.861183 3838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:29.861549 3838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:29.861711 3838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:29.862010 3838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:29.862360 3838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:29.862425 3838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:29.862468 3838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:29.862496 3838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:29.868566 3838 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:33413
I20260929 19:06:29.868608 3965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:33413 every 8 connection(s)
I20260929 19:06:29.868995 3838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:06:29.873137 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3838
I20260929 19:06:29.873235 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:06:29.873497 26939 external_mini_cluster.cc:1428] Setting key f0fc3f582fd571f6b12434c25a3dad38
I20260929 19:06:29.874573 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:42623
--builtin_ntp_servers=127.26.78.212:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 19:06:29.874680 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:29.869721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:32883 (local address 127.26.78.254:42623)
0929 19:06:29.869790 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:29.869792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:29.870057 (+ 265us) server_negotiation.cc:407] Connection header received
0929 19:06:29.870634 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:29.870635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:29.870650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:29.870712 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:29.871400 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.871937 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.872966 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.873205 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.873270 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.874388 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:29.874389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:29.874390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:29.874397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:29.874412 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:29.874417 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:29.874492 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:29.874567 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 19:06:29.874600 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:06:29.874898 3968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:29.869705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:42623 (local address 127.26.78.193:32883)
0929 19:06:29.869931 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:29.869968 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:29.870524 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:29.870804 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:29.870813 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:29.870821 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:29.871296 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.871299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.872045 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.872046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.872853 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.872855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.873338 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.873339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.873480 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.874196 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:06:29.874197 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:29.874239 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:06:29.874271 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:29.874491 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:29.874499 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:29.874502 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:29.874573 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 19:06:29.874680 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":130,"threads_started":1}
I20260929 19:06:29.875435 3966 heartbeater.cc:347] Connected to a master server at 127.26.78.254:42623
I20260929 19:06:29.875521 3966 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:29.875672 3966 heartbeater.cc:511] Master 127.26.78.254:42623 requested a full tablet report, sending...
I20260929 19:06:29.876304 3778 ts_manager.cc:194] Registered new tserver with Master: 7aaef2104f7441859463bcef25eb240a (127.26.78.193:33413)
I20260929 19:06:29.876946 3778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.193:32883
W20260929 19:06:29.957446 3969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:29.957641 3969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:29.957661 3969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:29.958839 3969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:29.958880 3969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:29.958905 3969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:06:29.960302 3969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:42623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:29.960569 3969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:29.960886 3969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:29.965953 3977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:29.965984 3974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:29.965998 3975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:29.966271 3969 server_base.cc:1034] running on GCE node
I20260929 19:06:29.966466 3969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:29.966704 3969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:29.967846 3969 hybrid_clock.cc:648] HybridClock initialized: now 1790708789967834 us; error 28 us; skew 500 ppm
I20260929 19:06:29.969092 3969 webserver.cc:460] Webserver started at http://127.26.78.194:41151/ using document root <none> and password file <none>
I20260929 19:06:29.969300 3969 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:29.969362 3969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:29.969465 3969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:29.970628 3969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "5ef3f745e4d14d398ccf2390f90b5280"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "a23ab9ac0aa8787d5f9b5170e6e32a3a"
server_key_iv: "a3d5eb32d169f5f59913a2c73bdfec34"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.970986 3969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "5ef3f745e4d14d398ccf2390f90b5280"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "a23ab9ac0aa8787d5f9b5170e6e32a3a"
server_key_iv: "a3d5eb32d169f5f59913a2c73bdfec34"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.972295 3969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:06:29.973032 3983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:29.973269 3969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:29.973383 3969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "5ef3f745e4d14d398ccf2390f90b5280"
format_stamp: "Formatted at 2026-09-29 19:06:29 on dist-test-slave-xz51"
server_key: "a23ab9ac0aa8787d5f9b5170e6e32a3a"
server_key_iv: "a3d5eb32d169f5f59913a2c73bdfec34"
server_key_version: "encryptionkey@0"
I20260929 19:06:29.973464 3969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:29.986167 3969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:29.986502 3969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:29.986658 3969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:29.986886 3969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:29.987221 3969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:29.987277 3969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:29.987329 3969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:29.987355 3969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:29.993175 3969 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:33229
I20260929 19:06:29.993217 4096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:33229 every 8 connection(s)
I20260929 19:06:29.993575 3969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:06:29.998760 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:29.994386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:47751 (local address 127.26.78.254:42623)
0929 19:06:29.994452 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:29.994455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:29.994706 (+ 251us) server_negotiation.cc:407] Connection header received
0929 19:06:29.995174 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:29.995176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:29.995186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:29.995239 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:29.995916 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.996404 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.997360 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.997545 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.997601 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.998476 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:29.998477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:29.998478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:29.998482 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:29.998495 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:29.998500 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:29.998554 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:29.998668 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 19:06:29.998698 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:06:29.998894 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:29.994379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:42623 (local address 127.26.78.194:47751)
0929 19:06:29.994591 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:29.994623 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:29.995091 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:29.995312 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:29.995316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:29.995324 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:29.995835 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.995841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.996480 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.996481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.997269 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:29.997271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:29.997620 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:29.997621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:29.997728 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:29.998340 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:06:29.998341 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:29.998370 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:06:29.998391 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:29.998566 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:29.998569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:29.998572 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:29.998625 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:06:29.998706 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260929 19:06:29.999382 4097 heartbeater.cc:347] Connected to a master server at 127.26.78.254:42623
I20260929 19:06:29.999469 4097 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:29.999605 4097 heartbeater.cc:511] Master 127.26.78.254:42623 requested a full tablet report, sending...
I20260929 19:06:29.999993 3778 ts_manager.cc:194] Registered new tserver with Master: 5ef3f745e4d14d398ccf2390f90b5280 (127.26.78.194:33229)
I20260929 19:06:30.000165 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 3969
I20260929 19:06:30.000245 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:06:30.000449 3778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.194:47751
I20260929 19:06:30.000468 26939 external_mini_cluster.cc:1428] Setting key 881093862082525775b17b5accc90010
I20260929 19:06:30.001371 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:42623
--builtin_ntp_servers=127.26.78.212:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 19:06:30.083665 4100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:30.083889 4100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:30.083911 4100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:30.085083 4100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:30.085125 4100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:30.085150 4100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:06:30.086611 4100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:42623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:30.086864 4100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:30.087185 4100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:30.092203 4105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.092274 4106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.092209 4108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:30.092454 4100 server_base.cc:1034] running on GCE node
I20260929 19:06:30.092654 4100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:30.092900 4100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:30.094038 4100 hybrid_clock.cc:648] HybridClock initialized: now 1790708790094027 us; error 31 us; skew 500 ppm
I20260929 19:06:30.095206 4100 webserver.cc:460] Webserver started at http://127.26.78.195:34449/ using document root <none> and password file <none>
I20260929 19:06:30.095407 4100 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:30.095448 4100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:30.095537 4100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:30.096546 4100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "71bd0eb50d6f4f53afb881df90596688"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "7038fcda34f82979c44c1da6cd9dae96"
server_key_iv: "a06217dbb190c36ea70401cb3f97821f"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.096843 4100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "71bd0eb50d6f4f53afb881df90596688"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "7038fcda34f82979c44c1da6cd9dae96"
server_key_iv: "a06217dbb190c36ea70401cb3f97821f"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.098147 4100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260929 19:06:30.098925 4114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.099138 4100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:06:30.099249 4100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "71bd0eb50d6f4f53afb881df90596688"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "7038fcda34f82979c44c1da6cd9dae96"
server_key_iv: "a06217dbb190c36ea70401cb3f97821f"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.099326 4100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:30.115094 4100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:30.115444 4100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:30.115602 4100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:30.115840 4100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:30.116190 4100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:30.116223 4100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.116246 4100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:30.116289 4100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.122236 4100 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:43653
I20260929 19:06:30.122273 4227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:43653 every 8 connection(s)
I20260929 19:06:30.122632 4100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:06:30.126369 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4100
I20260929 19:06:30.126492 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:06:30.126804 26939 external_mini_cluster.cc:1428] Setting key 5a12d6f01ed20353ee66378ce7b784bc
I20260929 19:06:30.128443 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.123409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:38431 (local address 127.26.78.254:42623)
0929 19:06:30.123456 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:30.123458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:30.123734 (+ 276us) server_negotiation.cc:407] Connection header received
0929 19:06:30.124203 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:30.124205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:30.124218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:30.124293 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:30.125003 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:30.125550 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:30.126709 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:30.127005 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:30.127072 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:30.128121 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:30.128122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:30.128123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:30.128128 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:30.128147 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:30.128153 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:30.128231 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:30.128332 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:06:30.128364 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:06:30.128641 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.123372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:42623 (local address 127.26.78.195:38431)
0929 19:06:30.123567 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:30.123611 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:30.124102 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:30.124368 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:30.124371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:30.124379 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:30.124906 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:30.124910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:30.125636 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:30.125637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:30.126613 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:30.126616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:30.127099 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:30.127100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:30.127267 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:30.127917 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:06:30.127917 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:30.127956 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:06:30.127983 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:30.128232 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:30.128239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:30.128242 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:30.128314 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:06:30.128412 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260929 19:06:30.129155 4228 heartbeater.cc:347] Connected to a master server at 127.26.78.254:42623
I20260929 19:06:30.129248 4228 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:30.129468 4228 heartbeater.cc:511] Master 127.26.78.254:42623 requested a full tablet report, sending...
I20260929 19:06:30.129933 3778 ts_manager.cc:194] Registered new tserver with Master: 71bd0eb50d6f4f53afb881df90596688 (127.26.78.195:43653)
I20260929 19:06:30.130297 3778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.195:38431
I20260929 19:06:30.139118 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 19:06:30.141187 3818 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:30.140650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.26.78.254:42623)
0929 19:06:30.140720 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:30.140721 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:30.140876 (+ 155us) server_negotiation.cc:407] Connection header received
0929 19:06:30.140919 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:30.140920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:30.140933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:30.140985 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:30.141105 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43694: BlockingRecv error: recv got EOF from 127.0.0.1:43694 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:06:30.141301 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3838
I20260929 19:06:30.146173 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3969
I20260929 19:06:30.150566 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4100
I20260929 19:06:30.154597 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 3747
2026-09-29T19:06:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (613 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T19:06:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:06:30Z Disabled control of system clock
I20260929 19:06:30.182099 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:45619
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:45619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 19:06:30.263706 4239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:30.263931 4239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:30.263953 4239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:30.265041 4239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:06:30.265074 4239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:30.265087 4239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:30.265100 4239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:06:30.265112 4239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:06:30.266515 4239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:45619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:45619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:30.266717 4239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:30.266968 4239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:30.271898 4245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.271922 4247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.271922 4244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:30.272235 4239 server_base.cc:1034] running on GCE node
I20260929 19:06:30.272495 4239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:30.272744 4239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:30.273895 4239 hybrid_clock.cc:648] HybridClock initialized: now 1790708790273891 us; error 47 us; skew 500 ppm
I20260929 19:06:30.275188 4239 webserver.cc:460] Webserver started at http://127.26.78.254:45961/ using document root <none> and password file <none>
I20260929 19:06:30.275436 4239 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:30.275509 4239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:30.275664 4239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:30.276835 4239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "b9c94be7a0904d84b80fce3ab7d1f45a"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "db90863b67fdf8430f3d0913510f7a5d"
server_key_iv: "dc9be1b52f3f3220ef28c6786d0f961d"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.277204 4239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "b9c94be7a0904d84b80fce3ab7d1f45a"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "db90863b67fdf8430f3d0913510f7a5d"
server_key_iv: "dc9be1b52f3f3220ef28c6786d0f961d"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.278630 4239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:06:30.279376 4253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.279546 4239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:30.279645 4239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "b9c94be7a0904d84b80fce3ab7d1f45a"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "db90863b67fdf8430f3d0913510f7a5d"
server_key_iv: "dc9be1b52f3f3220ef28c6786d0f961d"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.279729 4239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:30.283174 4239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:30.283439 4239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:30.283582 4239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:30.287585 4239 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:45619
I20260929 19:06:30.287613 4305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:45619 every 8 connection(s)
I20260929 19:06:30.287949 4239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:06:30.288600 4306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:30.291380 4306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a: Bootstrap starting.
I20260929 19:06:30.291973 4306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:30.292269 4306 log.cc:824] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:30.293103 4306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a: No bootstrap required, opened a new log
I20260929 19:06:30.294791 4306 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45619 } }
I20260929 19:06:30.294942 4306 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:30.294989 4306 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c94be7a0904d84b80fce3ab7d1f45a, State: Initialized, Role: FOLLOWER
I20260929 19:06:30.295140 4306 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45619 } }
I20260929 19:06:30.295200 4306 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:06:30.295239 4306 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:06:30.295287 4306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:30.295833 4306 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45619 } }
I20260929 19:06:30.295950 4306 leader_election.cc:302] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9c94be7a0904d84b80fce3ab7d1f45a; no voters:
I20260929 19:06:30.296123 4306 leader_election.cc:288] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:06:30.296185 4309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:30.296381 4306 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:06:30.296382 4309 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [term 1 LEADER]: Becoming Leader. State: Replica: b9c94be7a0904d84b80fce3ab7d1f45a, State: Running, Role: LEADER
I20260929 19:06:30.296450 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4239
I20260929 19:06:30.296545 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:06:30.296518 4309 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45619 } }
I20260929 19:06:30.296787 26939 external_mini_cluster.cc:1428] Setting key f1baac114dd7d269251723397b255077
I20260929 19:06:30.296919 4310 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45619 } } }
I20260929 19:06:30.297026 4310 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:30.297138 4311 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9c94be7a0904d84b80fce3ab7d1f45a. Latest consensus state: current_term: 1 leader_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c94be7a0904d84b80fce3ab7d1f45a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45619 } } }
I20260929 19:06:30.297206 4311 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:30.297966 4318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:06:30.298462 4318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:06:30.299454 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.297590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33784 (local address 127.26.78.254:45619)
0929 19:06:30.297942 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:30.297951 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:30.297986 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:06:30.298104 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:30.298111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:30.298157 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:30.298309 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:30.298995 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:30.298999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:30.298999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:30.299004 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:30.299051 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:30.299065 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:30.299142 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:30.299221 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 19:06:30.299276 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":61,"threads_started":1}
I20260929 19:06:30.300272 4318 catalog_manager.cc:1347] Generated new cluster ID: 3325582f44d7462f9c8865277b470029
I20260929 19:06:30.300325 4318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:06:30.327126 4318 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:06:30.327315 4318 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:06:30.329629 4318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9c94be7a0904d84b80fce3ab7d1f45a: Generated new TSK 0
I20260929 19:06:30.329809 4318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:06:30.333480 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45619
--builtin_ntp_servers=127.26.78.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 19:06:30.415400 4330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:30.415676 4330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:30.415714 4330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:30.416997 4330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:30.417043 4330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:30.417078 4330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:06:30.418598 4330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:30.418867 4330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:30.419196 4330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:30.424883 4335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.424944 4336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.424922 4338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:30.425101 4330 server_base.cc:1034] running on GCE node
I20260929 19:06:30.425352 4330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:30.425637 4330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:30.426803 4330 hybrid_clock.cc:648] HybridClock initialized: now 1790708790426765 us; error 50 us; skew 500 ppm
I20260929 19:06:30.428021 4330 webserver.cc:460] Webserver started at http://127.26.78.193:42327/ using document root <none> and password file <none>
I20260929 19:06:30.428300 4330 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:30.428390 4330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:30.428548 4330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:30.429594 4330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "58759cf6f5ed4cceadf88d70476b8b55"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "bab43075e776bd3bb9c2de9cdf782d27"
server_key_iv: "bc9f1126fbf545765da0185e00c2113a"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.429967 4330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "58759cf6f5ed4cceadf88d70476b8b55"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "bab43075e776bd3bb9c2de9cdf782d27"
server_key_iv: "bc9f1126fbf545765da0185e00c2113a"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.431154 4330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:06:30.431819 4344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.432000 4330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:06:30.432067 4330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "58759cf6f5ed4cceadf88d70476b8b55"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "bab43075e776bd3bb9c2de9cdf782d27"
server_key_iv: "bc9f1126fbf545765da0185e00c2113a"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.432118 4330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:30.435324 4330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:30.435542 4330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:30.435633 4330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:30.435786 4330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:30.436021 4330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:30.436051 4330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.436070 4330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:30.436084 4330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.441648 4330 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:36633
I20260929 19:06:30.441720 4457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:36633 every 8 connection(s)
I20260929 19:06:30.442072 4330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:06:30.445093 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.442905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:60915 (local address 127.26.78.254:45619)
0929 19:06:30.442986 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:30.442989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:30.443131 (+ 142us) server_negotiation.cc:407] Connection header received
0929 19:06:30.443600 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:30.443602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:30.443609 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:30.443668 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:30.444862 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:30.444863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:30.444863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:30.444867 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:30.444883 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:30.444886 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:30.444945 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:30.445028 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 19:06:30.445042 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 19:06:30.445261 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.442824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45619 (local address 127.26.78.193:60915)
0929 19:06:30.443011 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:30.443044 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:30.443512 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:30.443720 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:30.443723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:30.443731 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:30.444700 (+ 969us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:06:30.444700 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:30.444728 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:06:30.444756 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:30.444955 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:30.444959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:30.444962 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:30.445020 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:06:30.445068 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260929 19:06:30.445794 4458 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45619
I20260929 19:06:30.445912 4458 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:30.446099 4458 heartbeater.cc:511] Master 127.26.78.254:45619 requested a full tablet report, sending...
I20260929 19:06:30.446583 4270 ts_manager.cc:194] Registered new tserver with Master: 58759cf6f5ed4cceadf88d70476b8b55 (127.26.78.193:36633)
I20260929 19:06:30.448984 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4330
I20260929 19:06:30.449066 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:06:30.449270 26939 external_mini_cluster.cc:1428] Setting key 909e1a5fcd5c971193e8f4b6f552070d
I20260929 19:06:30.450352 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45619
--builtin_ntp_servers=127.26.78.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 19:06:30.531270 4461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:30.531491 4461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:30.531513 4461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:30.532655 4461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:30.532696 4461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:30.532749 4461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:06:30.534154 4461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:30.534447 4461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:30.534746 4461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:30.539712 4467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.539716 4466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.539734 4469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:30.539844 4461 server_base.cc:1034] running on GCE node
I20260929 19:06:30.540103 4461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:30.540364 4461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:30.541561 4461 hybrid_clock.cc:648] HybridClock initialized: now 1790708790541545 us; error 45 us; skew 500 ppm
I20260929 19:06:30.542927 4461 webserver.cc:460] Webserver started at http://127.26.78.194:38629/ using document root <none> and password file <none>
I20260929 19:06:30.543195 4461 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:30.543268 4461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:30.543406 4461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:30.544485 4461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "d6950949b95e4fb284737ec2094b227a"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "2a95e4a6413a4e24700bf6043b3b2d28"
server_key_iv: "5eb260024568c2c64709e9543216ac9b"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.544863 4461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "d6950949b95e4fb284737ec2094b227a"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "2a95e4a6413a4e24700bf6043b3b2d28"
server_key_iv: "5eb260024568c2c64709e9543216ac9b"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.546277 4461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:06:30.547078 4475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.547260 4461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:30.547360 4461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d6950949b95e4fb284737ec2094b227a"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "2a95e4a6413a4e24700bf6043b3b2d28"
server_key_iv: "5eb260024568c2c64709e9543216ac9b"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.547449 4461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:30.550830 4461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:30.551098 4461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:30.551235 4461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:30.551448 4461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:30.551759 4461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:30.551812 4461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.551858 4461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:30.551884 4461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.557578 4461 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:37613
I20260929 19:06:30.557621 4588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:37613 every 8 connection(s)
I20260929 19:06:30.558007 4461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:06:30.560779 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.558735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:58791 (local address 127.26.78.254:45619)
0929 19:06:30.558781 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:30.558783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:30.559037 (+ 254us) server_negotiation.cc:407] Connection header received
0929 19:06:30.559481 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:30.559481 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:30.559486 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:30.559549 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:30.560595 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:30.560596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:30.560597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:30.560601 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:30.560616 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:30.560620 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:30.560669 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:30.560716 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 19:06:30.560727 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:06:30.560993 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.558707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45619 (local address 127.26.78.194:58791)
0929 19:06:30.558908 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:30.558948 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:30.559393 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:30.559600 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:30.559604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:30.559611 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:30.560455 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:06:30.560455 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:30.560483 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:06:30.560510 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:30.560664 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:30.560667 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:30.560669 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:06:30.560716 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:06:30.560761 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260929 19:06:30.561545 4589 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45619
I20260929 19:06:30.561630 4589 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:30.561810 4589 heartbeater.cc:511] Master 127.26.78.254:45619 requested a full tablet report, sending...
I20260929 19:06:30.562237 4270 ts_manager.cc:194] Registered new tserver with Master: d6950949b95e4fb284737ec2094b227a (127.26.78.194:37613)
I20260929 19:06:30.565165 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4461
I20260929 19:06:30.565250 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:06:30.565414 26939 external_mini_cluster.cc:1428] Setting key 00bfce8c6b10640e5a21dc2e11110702
I20260929 19:06:30.566256 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45619
--builtin_ntp_servers=127.26.78.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 19:06:30.647117 4592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:30.647332 4592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:30.647354 4592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:30.648443 4592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:30.648480 4592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:30.648505 4592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:06:30.649748 4592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:30.650046 4592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:30.650308 4592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:30.655490 4597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:30.655519 4592 server_base.cc:1034] running on GCE node
W20260929 19:06:30.655531 4598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:30.655501 4600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:30.655884 4592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:30.656137 4592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:30.657326 4592 hybrid_clock.cc:648] HybridClock initialized: now 1790708790657312 us; error 26 us; skew 500 ppm
I20260929 19:06:30.658644 4592 webserver.cc:460] Webserver started at http://127.26.78.195:38853/ using document root <none> and password file <none>
I20260929 19:06:30.658941 4592 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:30.659016 4592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:30.659166 4592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:30.660230 4592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "5184c9d500004e61965f0fc6cbace950"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "3133855f2ef647505af386ba160090b0"
server_key_iv: "8782ea7e13620608a6ebd612c05dad0f"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.660601 4592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "5184c9d500004e61965f0fc6cbace950"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "3133855f2ef647505af386ba160090b0"
server_key_iv: "8782ea7e13620608a6ebd612c05dad0f"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.661935 4592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:06:30.662724 4606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.662899 4592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:30.662989 4592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "5184c9d500004e61965f0fc6cbace950"
format_stamp: "Formatted at 2026-09-29 19:06:30 on dist-test-slave-xz51"
server_key: "3133855f2ef647505af386ba160090b0"
server_key_iv: "8782ea7e13620608a6ebd612c05dad0f"
server_key_version: "encryptionkey@0"
I20260929 19:06:30.663074 4592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:30.666500 4592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:30.666781 4592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:30.666927 4592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:30.667140 4592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:30.667448 4592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:30.667500 4592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.667539 4592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:30.667567 4592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:30.673403 4592 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:44887
I20260929 19:06:30.673444 4719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:44887 every 8 connection(s)
I20260929 19:06:30.673760 4592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:06:30.676820 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.674526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:33017 (local address 127.26.78.254:45619)
0929 19:06:30.674568 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:30.674572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:30.674817 (+ 245us) server_negotiation.cc:407] Connection header received
0929 19:06:30.675327 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:30.675328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:30.675334 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:30.675421 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:30.676589 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:30.676590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:30.676591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:06:30.676595 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:30.676609 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:06:30.676613 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:30.676664 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:30.676753 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 19:06:30.676765 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:06:30.676971 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:30.674507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45619 (local address 127.26.78.195:33017)
0929 19:06:30.674689 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:30.674729 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:30.675187 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:30.675498 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:30.675502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:30.675509 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:30.676444 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:06:30.676444 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:30.676472 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:06:30.676499 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:30.676675 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:30.676679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:30.676681 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:06:30.676739 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:06:30.676787 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260929 19:06:30.677461 4720 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45619
I20260929 19:06:30.677570 4720 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:30.677718 4720 heartbeater.cc:511] Master 127.26.78.254:45619 requested a full tablet report, sending...
I20260929 19:06:30.678144 4270 ts_manager.cc:194] Registered new tserver with Master: 5184c9d500004e61965f0fc6cbace950 (127.26.78.195:44887)
I20260929 19:06:30.680986 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4592
I20260929 19:06:30.681056 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:06:30.681248 26939 external_mini_cluster.cc:1428] Setting key 1b19af7504dc6d7a70d9ac903c2aba9a
I20260929 19:06:30.682204 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 19:06:30.684121 4325 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:06:30.683571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.26.78.254:45619)
0929 19:06:30.683609 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:30.683611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:30.683783 (+ 172us) server_negotiation.cc:407] Connection header received
0929 19:06:30.683871 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:30.683872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:30.683877 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:30.683926 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:30.684036 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33800: BlockingRecv error: recv got EOF from 127.0.0.1:33800 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:06:30.684224 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4330
I20260929 19:06:30.689410 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4461
I20260929 19:06:30.693898 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4592
I20260929 19:06:30.698410 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4239
2026-09-29T19:06:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (543 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:06:30 dist-test-slave-xz51 krb5kdc[4730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:06:30 dist-test-slave-xz51 krb5kdc[4730](info): set up 2 sockets
Sep 29 19:06:30 dist-test-slave-xz51 krb5kdc[4730](info): commencing operation
krb5kdc: starting...
W20260929 19:06:31.742223 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:06:31 dist-test-slave-xz51 krb5kdc[4730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:06:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:06:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:06:31.864455 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40055
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:40055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:31.945941 4743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:31.946141 4743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:31.946162 4743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:31.947357 4743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:06:31.947397 4743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:31.947412 4743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:31.947424 4743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:06:31.947438 4743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:06:31.949015 4743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:40055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:31.949268 4743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:31.949502 4743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:06:31.954303 4743 server_base.cc:1034] running on GCE node
W20260929 19:06:31.954276 4749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:31.954285 4751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:31.954294 4748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:31.954737 4743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:31.955029 4743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:31.956175 4743 hybrid_clock.cc:648] HybridClock initialized: now 1790708791956156 us; error 36 us; skew 500 ppm
Sep 29 19:06:31 dist-test-slave-xz51 krb5kdc[4730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708791, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:31.958777 4743 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:06:31.959205 4743 webserver.cc:460] Webserver started at http://127.26.78.254:41629/ using document root <none> and password file <none>
I20260929 19:06:31.959445 4743 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:31.959491 4743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:31.959637 4743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:31.960676 4743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "cf3278be52684c85a21d0efb515e92db"
format_stamp: "Formatted at 2026-09-29 19:06:31 on dist-test-slave-xz51"
server_key: "18cd9626174adc1f09431c7bf8929f31"
server_key_iv: "95c60c807b1e948f9424026fb4c04c73"
server_key_version: "encryptionkey@0"
I20260929 19:06:31.961023 4743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "cf3278be52684c85a21d0efb515e92db"
format_stamp: "Formatted at 2026-09-29 19:06:31 on dist-test-slave-xz51"
server_key: "18cd9626174adc1f09431c7bf8929f31"
server_key_iv: "95c60c807b1e948f9424026fb4c04c73"
server_key_version: "encryptionkey@0"
I20260929 19:06:31.962401 4743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:06:31.963128 4758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:31.963325 4743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:06:31.963416 4743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "cf3278be52684c85a21d0efb515e92db"
format_stamp: "Formatted at 2026-09-29 19:06:31 on dist-test-slave-xz51"
server_key: "18cd9626174adc1f09431c7bf8929f31"
server_key_iv: "95c60c807b1e948f9424026fb4c04c73"
server_key_version: "encryptionkey@0"
I20260929 19:06:31.963488 4743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:31.975776 4743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:31.977612 4743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:31.977771 4743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:31.981555 4743 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:40055
I20260929 19:06:31.981637 4810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:40055 every 8 connection(s)
I20260929 19:06:31.982046 4743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:06:31.983042 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:31.985297 4811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db: Bootstrap starting.
I20260929 19:06:31.985958 4811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:31.986243 4811 log.cc:824] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:31.986936 4811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db: No bootstrap required, opened a new log
I20260929 19:06:31.988509 4811 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3278be52684c85a21d0efb515e92db" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40055 } }
I20260929 19:06:31.988672 4811 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:31.988720 4811 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf3278be52684c85a21d0efb515e92db, State: Initialized, Role: FOLLOWER
I20260929 19:06:31.988901 4811 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3278be52684c85a21d0efb515e92db" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40055 } }
I20260929 19:06:31.989004 4811 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:06:31.989048 4811 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:06:31.989097 4811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:31.989529 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4743
I20260929 19:06:31.989637 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:06:31.989933 26939 external_mini_cluster.cc:1428] Setting key 32e7bc0c3d60f63523693651d2b8b51b
I20260929 19:06:31.990234 4811 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3278be52684c85a21d0efb515e92db" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40055 } }
I20260929 19:06:31.990366 4811 leader_election.cc:302] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf3278be52684c85a21d0efb515e92db; no voters:
I20260929 19:06:31.990531 4811 leader_election.cc:288] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:06:31.990568 4815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:31.990783 4815 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [term 1 LEADER]: Becoming Leader. State: Replica: cf3278be52684c85a21d0efb515e92db, State: Running, Role: LEADER
I20260929 19:06:31.990887 4811 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:06:31.990931 4815 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3278be52684c85a21d0efb515e92db" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40055 } }
I20260929 19:06:31.991335 4818 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf3278be52684c85a21d0efb515e92db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3278be52684c85a21d0efb515e92db" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40055 } } }
I20260929 19:06:31.991444 4818 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [sys.catalog]: This master's current role is: LEADER
I20260929 19:06:31.991698 4817 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf3278be52684c85a21d0efb515e92db. Latest consensus state: current_term: 1 leader_uuid: "cf3278be52684c85a21d0efb515e92db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3278be52684c85a21d0efb515e92db" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40055 } } }
I20260929 19:06:31.992070 4817 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db [sys.catalog]: This master's current role is: LEADER
W20260929 19:06:31.992753 4832 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:06:31.992807 4832 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:06:31.992868 4823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:06:31.993014 4823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:06:31 dist-test-slave-xz51 krb5kdc[4730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:31.997409 4823 catalog_manager.cc:1347] Generated new cluster ID: d34502437a7647aaa38d1bcc253efc92
I20260929 19:06:31.997462 4823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:06:32.001032 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:31.990441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39770 (local address 127.26.78.254:40055)
0929 19:06:31.990622 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:31.990633 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:31.990692 (+ 59us) server_negotiation.cc:407] Connection header received
0929 19:06:31.990861 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:31.990868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:31.990927 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:31.991784 (+ 857us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:31.991920 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:31.993277 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:31.994197 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:31.994622 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:31.994747 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:31.997063 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:31.997068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:31.997068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:31.997080 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:31.999349 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:31.999749 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:31.999752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:31.999753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:31.999905 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.000111 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.000112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.000113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.000438 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:32.000598 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.000669 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.000810 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260929 19:06:32.012414 4823 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:06:32.013175 4823 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:06:32.019644 4823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db: Generated new TSK 0
I20260929 19:06:32.019869 4823 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:06:32.044991 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40055
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:32.127005 4837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:32.127206 4837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:32.127229 4837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:32.128756 4837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:32.128814 4837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:32.128844 4837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:06:32.130434 4837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:32.130683 4837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:32.130926 4837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:32.136134 4845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:32.136188 4842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:32.136123 4843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:32.136363 4837 server_base.cc:1034] running on GCE node
I20260929 19:06:32.136571 4837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:32.136827 4837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:32.137979 4837 hybrid_clock.cc:648] HybridClock initialized: now 1790708792137966 us; error 29 us; skew 500 ppm
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[4730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708792, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:32.140647 4837 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:06:32.141090 4837 webserver.cc:460] Webserver started at http://127.26.78.193:42961/ using document root <none> and password file <none>
I20260929 19:06:32.141314 4837 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:32.141360 4837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:32.141489 4837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:32.142581 4837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "234f2d4e0ee043eaad4521b5b83f5c46"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "1c061c16a26b41f13f5914caf188e232"
server_key_iv: "45e9e50172a30af944c451aef30608ab"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.142931 4837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "234f2d4e0ee043eaad4521b5b83f5c46"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "1c061c16a26b41f13f5914caf188e232"
server_key_iv: "45e9e50172a30af944c451aef30608ab"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.144236 4837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:06:32.144994 4852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.145210 4837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:06:32.145306 4837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "234f2d4e0ee043eaad4521b5b83f5c46"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "1c061c16a26b41f13f5914caf188e232"
server_key_iv: "45e9e50172a30af944c451aef30608ab"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.145362 4837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:32.155920 4837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:32.158417 4837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:32.158535 4837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:32.158716 4837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:32.159044 4837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:32.159077 4837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.159123 4837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:32.159148 4837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.164796 4837 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:41199
I20260929 19:06:32.164899 4965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:41199 every 8 connection(s)
I20260929 19:06:32.165203 4837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:06:32.171015 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4837
I20260929 19:06:32.171121 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:06:32.171348 26939 external_mini_cluster.cc:1428] Setting key 362c363c88416bdb15733ee0dba2c818
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[4730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708792, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:32.175343 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.165975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40055 (local address 127.26.78.193:59717)
0929 19:06:32.166148 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.166184 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.166652 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.166862 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.166866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.167549 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:32.167743 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.167748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.168421 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.168422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.169236 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.169237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.169599 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.169600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.169701 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.170273 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:32.170281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:32.171995 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:32.174019 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:32.174025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:32.174027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:32.174239 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:32.174450 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:32.174451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:32.174451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:32.174549 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:32.174839 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:32.174845 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:32.174964 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.175049 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.175170 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260929 19:06:32.175343 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.165964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:59717 (local address 127.26.78.254:40055)
0929 19:06:32.166037 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.166039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.166266 (+ 227us) server_negotiation.cc:407] Connection header received
0929 19:06:32.166734 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.166736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.166750 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.166804 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:32.167830 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.168335 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.169316 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.169526 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.169580 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.172087 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:32.172088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:32.172089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:32.172097 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:32.173938 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.174316 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.174317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.174317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.174381 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.174613 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.174614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.174614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.174775 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:32.174845 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.175093 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.175201 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:06:32.175877 4966 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40055
I20260929 19:06:32.175989 4966 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:32.176170 4966 heartbeater.cc:511] Master 127.26.78.254:40055 requested a full tablet report, sending...
I20260929 19:06:32.176752 4775 ts_manager.cc:194] Registered new tserver with Master: 234f2d4e0ee043eaad4521b5b83f5c46 (127.26.78.193:41199)
I20260929 19:06:32.177361 4775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:59717
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:06:32.197044 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40055
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:32.280375 4971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:32.280623 4971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:32.280668 4971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:32.281915 4971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:32.281971 4971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:32.282016 4971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:06:32.283527 4971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:32.283807 4971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:32.284047 4971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:32.289402 4977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:32.289491 4976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:32.289569 4971 server_base.cc:1034] running on GCE node
W20260929 19:06:32.289402 4979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:32.289956 4971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:32.290249 4971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:32.291424 4971 hybrid_clock.cc:648] HybridClock initialized: now 1790708792291406 us; error 46 us; skew 500 ppm
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[4730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708792, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:32.294476 4971 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:06:32.294926 4971 webserver.cc:460] Webserver started at http://127.26.78.194:38023/ using document root <none> and password file <none>
I20260929 19:06:32.295171 4971 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:32.295238 4971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:32.295387 4971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:32.296514 4971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "8390d0fbdf2b45878a929300db4c26b9"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "8f96b2b5c224b6084b2be47c5be4d707"
server_key_iv: "d74008d3ed730d0cc4cee82586a5e759"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.296864 4971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "8390d0fbdf2b45878a929300db4c26b9"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "8f96b2b5c224b6084b2be47c5be4d707"
server_key_iv: "d74008d3ed730d0cc4cee82586a5e759"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.298287 4971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:06:32.299022 4986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.299183 4971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:06:32.299284 4971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "8390d0fbdf2b45878a929300db4c26b9"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "8f96b2b5c224b6084b2be47c5be4d707"
server_key_iv: "d74008d3ed730d0cc4cee82586a5e759"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.299367 4971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:32.312081 4971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:32.314534 4971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:32.314769 4971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:32.315171 4971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:32.315526 4971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:32.315580 4971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.315615 4971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:32.315654 4971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.321652 4971 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:44481
I20260929 19:06:32.321789 5099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:44481 every 8 connection(s)
I20260929 19:06:32.322122 4971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[4730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708792, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:32.331825 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.322942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:46345 (local address 127.26.78.254:40055)
0929 19:06:32.323047 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.323049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.323252 (+ 203us) server_negotiation.cc:407] Connection header received
0929 19:06:32.323750 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.323751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.323762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.323835 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:32.324817 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.325335 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.326345 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.326551 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.326616 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.328824 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:32.328826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:32.328826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:32.328834 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:32.330343 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.330825 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.330826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.330827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.330886 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.331185 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.331186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.331186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.331298 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:32.331354 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.331680 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.331757 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 19:06:32.332031 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.322888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40055 (local address 127.26.78.194:46345)
0929 19:06:32.323080 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.323122 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.323621 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.323894 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.323897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.324461 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:32.324673 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.324678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.325422 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.325423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.326253 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.326255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.326631 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.326632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.326733 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.327201 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:32.327209 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:32.328728 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:32.330439 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:32.330446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:32.330448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:32.330745 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:32.330973 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:32.330974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:32.330974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:32.331099 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:32.331383 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:32.331400 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:32.331527 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.331623 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.331781 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":124,"threads_started":1}
I20260929 19:06:32.332211 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 4971
I20260929 19:06:32.332322 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:06:32.332571 26939 external_mini_cluster.cc:1428] Setting key a5bc989fe80e9c226101ce5671cefd2d
I20260929 19:06:32.332590 5100 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40055
I20260929 19:06:32.332683 5100 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:32.332898 5100 heartbeater.cc:511] Master 127.26.78.254:40055 requested a full tablet report, sending...
I20260929 19:06:32.333312 4775 ts_manager.cc:194] Registered new tserver with Master: 8390d0fbdf2b45878a929300db4c26b9 (127.26.78.194:44481)
I20260929 19:06:32.333693 4775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:46345
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:06:32.357040 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40055
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:06:32.439651 5105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:06:32.439858 5105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:06:32.439890 5105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:06:32.440989 5105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:06:32.441027 5105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:06:32.441053 5105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:06:32.442541 5105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:06:32.442736 5105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:06:32.442922 5105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:06:32.447988 5113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:32.447995 5111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:06:32.448109 5110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:06:32.448529 5105 server_base.cc:1034] running on GCE node
I20260929 19:06:32.448758 5105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:06:32.449040 5105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:06:32.450182 5105 hybrid_clock.cc:648] HybridClock initialized: now 1790708792450159 us; error 40 us; skew 500 ppm
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[4730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708792, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:06:32.452854 5105 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:06:32.453334 5105 webserver.cc:460] Webserver started at http://127.26.78.195:40915/ using document root <none> and password file <none>
I20260929 19:06:32.453578 5105 fs_manager.cc:359] Metadata directory not provided
I20260929 19:06:32.453646 5105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:06:32.453791 5105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:06:32.454950 5105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "dca6cc23dc32473595146ec139d3b0d6"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "bb3cd665a0d03bd94ff28ce46330e104"
server_key_iv: "e99907b6a37389fe47847be82f246d53"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.455322 5105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "dca6cc23dc32473595146ec139d3b0d6"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "bb3cd665a0d03bd94ff28ce46330e104"
server_key_iv: "e99907b6a37389fe47847be82f246d53"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.456615 5105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:06:32.457345 5120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.457515 5105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:06:32.457614 5105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "dca6cc23dc32473595146ec139d3b0d6"
format_stamp: "Formatted at 2026-09-29 19:06:32 on dist-test-slave-xz51"
server_key: "bb3cd665a0d03bd94ff28ce46330e104"
server_key_iv: "e99907b6a37389fe47847be82f246d53"
server_key_version: "encryptionkey@0"
I20260929 19:06:32.457692 5105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:06:32.472934 5105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:06:32.474822 5105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:06:32.474997 5105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:06:32.475227 5105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:06:32.475545 5105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:06:32.475600 5105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.475641 5105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:06:32.475669 5105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:06:32.481813 5105 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:37217
I20260929 19:06:32.481876 5233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:37217 every 8 connection(s)
I20260929 19:06:32.482203 5105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:06:32.482482 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 5105
I20260929 19:06:32.482591 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:06:32.482825 26939 external_mini_cluster.cc:1428] Setting key 9116fc4f8afa11f365d8a6ce491acb2e
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[4730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708792, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:06:32.492226 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.483162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:35933 (local address 127.26.78.254:40055)
0929 19:06:32.483219 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.483221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.483547 (+ 326us) server_negotiation.cc:407] Connection header received
0929 19:06:32.484204 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.484205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.484214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.484278 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:32.485235 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.485772 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.486852 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.487070 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.487127 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.489223 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:32.489225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:32.489225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:32.489233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:32.490684 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.491114 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.491115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.491116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.491177 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.491449 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.491450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.491450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.491620 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:32.491689 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.491999 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.492094 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:06:32.492270 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.483128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40055 (local address 127.26.78.195:35933)
0929 19:06:32.483409 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.483454 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.484087 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.484334 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.484338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.484957 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:06:32.485144 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.485149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.485907 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.485908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.486733 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.486735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.487147 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.487148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.487249 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.487703 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:06:32.487710 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:06:32.489131 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:06:32.490752 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:32.490758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:32.490760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:32.491032 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:32.491235 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:06:32.491236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:06:32.491236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:06:32.491360 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:06:32.491680 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:06:32.491685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:06:32.491802 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.491946 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.492110 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":99,"threads_started":1}
I20260929 19:06:32.492740 5234 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40055
I20260929 19:06:32.492837 5234 heartbeater.cc:464] Registering TS with master...
I20260929 19:06:32.492977 5234 heartbeater.cc:511] Master 127.26.78.254:40055 requested a full tablet report, sending...
I20260929 19:06:32.493346 4775 ts_manager.cc:194] Registered new tserver with Master: dca6cc23dc32473595146ec139d3b0d6 (127.26.78.195:37217)
I20260929 19:06:32.493681 4775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:35933
I20260929 19:06:32.494786 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:06:32.502429 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.496164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.26.78.254:40055)
0929 19:06:32.496304 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.496306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.496400 (+ 94us) server_negotiation.cc:407] Connection header received
0929 19:06:32.496467 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.496468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.496478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.496532 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:06:32.497129 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.497645 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.498470 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.498668 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.498762 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.499953 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:06:32.499955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:06:32.499955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:06:32.499962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:06:32.501120 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.501480 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.501481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.501481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.501542 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:06:32.501753 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:06:32.501754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:06:32.501754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:06:32.501966 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:06:32.502069 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.502260 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.502331 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260929 19:06:32.503981 4775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:06:32.505455 4775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:06:32.511803 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.507626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:41199 (local address 127.0.0.1:34580)
0929 19:06:32.507795 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.507859 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.507925 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.508324 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.508324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.508326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:32.508402 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.508404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.509723 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.509724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.510801 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.510803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.511503 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.511504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.511593 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.511607 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.511676 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.511727 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":19,"thread_start_us":71,"threads_started":1}
I20260929 19:06:32.512130 5246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.507819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.26.78.193:41199)
0929 19:06:32.508151 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.508155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.508163 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:06:32.508204 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.508206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.508222 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.508324 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:32.508494 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.509614 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.510892 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.511386 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.511503 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.511565 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.512004 (+ 439us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.512048 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":85,"threads_started":1}
I20260929 19:06:32.512666 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.507782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49334 (local address 127.26.78.195:37217)
0929 19:06:32.508057 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.508061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.508070 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:32.508112 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.508114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.508130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.508232 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:32.508409 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.509316 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.511601 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.512324 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.512385 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.512467 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.512550 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.512587 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":86,"threads_started":1}
I20260929 19:06:32.512667 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.507626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:37217 (local address 127.0.0.1:49334)
0929 19:06:32.507986 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.507995 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.508042 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.508247 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.508249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.508251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:32.508322 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.508324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.509460 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.509461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.510788 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.510791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.512390 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.512391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.512466 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.512469 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.512536 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.512576 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":4,"spinlock_wait_cycles":384,"thread_start_us":66,"threads_started":1}
I20260929 19:06:32.513007 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.507636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:44481 (local address 127.0.0.1:54234)
0929 19:06:32.508019 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.508026 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.508075 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.508315 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.508316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.508319 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:32.508455 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.508457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.510896 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.510896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.511581 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.511584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.512766 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.512767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.512841 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.512844 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.512897 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.512933 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":52,"threads_started":1}
I20260929 19:06:32.513063 5247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.507882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.26.78.194:44481)
0929 19:06:32.508022 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.508027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.508069 (+ 42us) server_negotiation.cc:407] Connection header received
0929 19:06:32.508140 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.508142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.508159 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.508232 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:32.508537 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.509621 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.511675 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.512617 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.512678 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.512729 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.512939 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.512987 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260929 19:06:32.512938 4900 tablet_service.cc:1467] Processing CreateTablet for tablet 4d801325b5a347a28e618b1f6eed879b (DEFAULT_TABLE table=test-table [id=3dc5d249b4a74819b828cb72591025f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:32.513324 4900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d801325b5a347a28e618b1f6eed879b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:32.513625 5168 tablet_service.cc:1467] Processing CreateTablet for tablet 4d801325b5a347a28e618b1f6eed879b (DEFAULT_TABLE table=test-table [id=3dc5d249b4a74819b828cb72591025f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:32.513936 5168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d801325b5a347a28e618b1f6eed879b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:32.516131 5248 tablet_bootstrap.cc:492] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46: Bootstrap starting.
I20260929 19:06:32.516495 5249 tablet_bootstrap.cc:492] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Bootstrap starting.
I20260929 19:06:32.516871 5248 tablet_bootstrap.cc:654] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:32.516862 5034 tablet_service.cc:1467] Processing CreateTablet for tablet 4d801325b5a347a28e618b1f6eed879b (DEFAULT_TABLE table=test-table [id=3dc5d249b4a74819b828cb72591025f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:06:32.517184 5034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d801325b5a347a28e618b1f6eed879b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:06:32.517398 5249 tablet_bootstrap.cc:654] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:32.517589 5248 log.cc:824] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:32.518040 5249 log.cc:824] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:32.518517 5248 tablet_bootstrap.cc:492] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46: No bootstrap required, opened a new log
I20260929 19:06:32.518589 5248 ts_tablet_manager.cc:1397] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260929 19:06:32.518740 5249 tablet_bootstrap.cc:492] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: No bootstrap required, opened a new log
I20260929 19:06:32.518805 5249 ts_tablet_manager.cc:1397] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:06:32.520054 5252 tablet_bootstrap.cc:492] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Bootstrap starting.
I20260929 19:06:32.520553 5248 raft_consensus.cc:348] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.520740 5248 raft_consensus.cc:374] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:32.520684 5249 raft_consensus.cc:348] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.520776 5248 raft_consensus.cc:729] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 234f2d4e0ee043eaad4521b5b83f5c46, State: Initialized, Role: FOLLOWER
I20260929 19:06:32.520816 5249 raft_consensus.cc:374] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:32.520848 5249 raft_consensus.cc:729] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca6cc23dc32473595146ec139d3b0d6, State: Initialized, Role: FOLLOWER
I20260929 19:06:32.520988 5252 tablet_bootstrap.cc:654] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Neither blocks nor log segments found. Creating new log.
I20260929 19:06:32.520952 5248 consensus_queue.cc:260] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.520962 5249 consensus_queue.cc:260] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.521258 5249 ts_tablet_manager.cc:1428] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:06:32.521260 5248 ts_tablet_manager.cc:1428] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:06:32.521382 4966 heartbeater.cc:503] Master 127.26.78.254:40055 was elected leader, sending a full tablet report...
I20260929 19:06:32.521395 5234 heartbeater.cc:503] Master 127.26.78.254:40055 was elected leader, sending a full tablet report...
I20260929 19:06:32.521849 5252 log.cc:824] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Log is configured to *not* fsync() on all Append() calls
I20260929 19:06:32.522750 5252 tablet_bootstrap.cc:492] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: No bootstrap required, opened a new log
I20260929 19:06:32.522820 5252 ts_tablet_manager.cc:1397] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:06:32.524119 5252 raft_consensus.cc:348] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.524238 5252 raft_consensus.cc:374] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:06:32.524260 5252 raft_consensus.cc:729] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8390d0fbdf2b45878a929300db4c26b9, State: Initialized, Role: FOLLOWER
I20260929 19:06:32.524401 5252 consensus_queue.cc:260] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.524688 5252 ts_tablet_manager.cc:1428] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:06:32.524739 5100 heartbeater.cc:503] Master 127.26.78.254:40055 was elected leader, sending a full tablet report...
I20260929 19:06:32.554414 5254 raft_consensus.cc:482] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:06:32.554565 5254 raft_consensus.cc:504] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.554991 5254 leader_election.cc:288] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dca6cc23dc32473595146ec139d3b0d6 (127.26.78.195:37217), 8390d0fbdf2b45878a929300db4c26b9 (127.26.78.194:44481)
I20260929 19:06:32.558000 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.555273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:39603 (local address 127.26.78.195:37217)
0929 19:06:32.555322 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.555324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.555346 (+ 22us) server_negotiation.cc:407] Connection header received
0929 19:06:32.555394 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.555394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.555405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.555475 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:32.555666 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.556211 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.557230 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.557604 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.557660 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.557691 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.557886 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.557927 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:06:32.558007 5247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.555221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:56039 (local address 127.26.78.194:44481)
0929 19:06:32.555279 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.555282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.555521 (+ 239us) server_negotiation.cc:407] Connection header received
0929 19:06:32.555564 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.555565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.555578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.555630 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:06:32.555808 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.556291 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.557237 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.557610 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.557700 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.557709 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.557901 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.557941 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:06:32.558007 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.555164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:37217 (local address 127.26.78.193:39603)
0929 19:06:32.555251 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.555264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.555312 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.555506 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.555507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.555509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:32.555592 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.555594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.556296 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.556297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.557137 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.557140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.557689 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.557690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.557769 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.557772 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.557881 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.557924 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":28}
I20260929 19:06:32.558017 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.555164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:44481 (local address 127.26.78.193:56039)
0929 19:06:32.555435 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.555446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:06:32.555504 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:06:32.555667 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:06:32.555668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:06:32.555670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:06:32.555735 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.555736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.556385 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.556385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.557139 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:06:32.557141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.557721 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.557722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:06:32.557781 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.557784 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:06:32.557887 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 19:06:32.557928 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":52,"thread_start_us":129,"threads_started":1}
I20260929 19:06:32.558557 5054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d801325b5a347a28e618b1f6eed879b" candidate_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8390d0fbdf2b45878a929300db4c26b9" is_pre_election: true
I20260929 19:06:32.558562 5188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d801325b5a347a28e618b1f6eed879b" candidate_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca6cc23dc32473595146ec139d3b0d6" is_pre_election: true
I20260929 19:06:32.558710 5054 raft_consensus.cc:2454] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 234f2d4e0ee043eaad4521b5b83f5c46 in term 0.
I20260929 19:06:32.558732 5188 raft_consensus.cc:2454] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 234f2d4e0ee043eaad4521b5b83f5c46 in term 0.
I20260929 19:06:32.558946 4854 leader_election.cc:302] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234f2d4e0ee043eaad4521b5b83f5c46, 8390d0fbdf2b45878a929300db4c26b9; no voters:
I20260929 19:06:32.559118 5254 raft_consensus.cc:2781] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:06:32.559165 5254 raft_consensus.cc:482] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:06:32.559181 5254 raft_consensus.cc:3037] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:32.560048 5254 raft_consensus.cc:504] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.560197 5254 leader_election.cc:288] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [CANDIDATE]: Term 1 election: Requested vote from peers dca6cc23dc32473595146ec139d3b0d6 (127.26.78.195:37217), 8390d0fbdf2b45878a929300db4c26b9 (127.26.78.194:44481)
I20260929 19:06:32.560357 5054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d801325b5a347a28e618b1f6eed879b" candidate_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8390d0fbdf2b45878a929300db4c26b9"
I20260929 19:06:32.560391 5188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d801325b5a347a28e618b1f6eed879b" candidate_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca6cc23dc32473595146ec139d3b0d6"
I20260929 19:06:32.560461 5054 raft_consensus.cc:3037] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:32.560461 5188 raft_consensus.cc:3037] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:06:32.561231 5054 raft_consensus.cc:2454] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 234f2d4e0ee043eaad4521b5b83f5c46 in term 1.
I20260929 19:06:32.561369 5188 raft_consensus.cc:2454] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 234f2d4e0ee043eaad4521b5b83f5c46 in term 1.
I20260929 19:06:32.561455 4854 leader_election.cc:302] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234f2d4e0ee043eaad4521b5b83f5c46, 8390d0fbdf2b45878a929300db4c26b9; no voters:
I20260929 19:06:32.561558 5254 raft_consensus.cc:2781] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:06:32.561725 5254 raft_consensus.cc:686] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [term 1 LEADER]: Becoming Leader. State: Replica: 234f2d4e0ee043eaad4521b5b83f5c46, State: Running, Role: LEADER
I20260929 19:06:32.561874 5254 consensus_queue.cc:237] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } }
I20260929 19:06:32.562784 4775 catalog_manager.cc:5495] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 234f2d4e0ee043eaad4521b5b83f5c46 (127.26.78.193). New cstate: current_term: 1 leader_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f2d4e0ee043eaad4521b5b83f5c46" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 } health_report { overall_health: UNKNOWN } } }
I20260929 19:06:32.570549 5246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.568289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.26.78.193:41199)
0929 19:06:32.568342 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:06:32.568344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:06:32.568353 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:06:32.568412 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:06:32.568412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:06:32.568429 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:06:32.568491 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:06:32.568621 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.569075 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.569770 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:06:32.570022 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:06:32.570074 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:06:32.570247 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:06:32.570351 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:06:32.570435 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:06:32.570453 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 19:06:32.570472 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 19:06:32.572463 5188 raft_consensus.cc:1261] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 1 FOLLOWER]: Refusing update from remote peer 234f2d4e0ee043eaad4521b5b83f5c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:06:32.572480 5054 raft_consensus.cc:1261] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 1 FOLLOWER]: Refusing update from remote peer 234f2d4e0ee043eaad4521b5b83f5c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260929 19:06:32.572742 5101 tablet.cc:2369] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:06:32.572830 5254 consensus_queue.cc:1035] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8390d0fbdf2b45878a929300db4c26b9" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 44481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:32.572975 5258 consensus_queue.cc:1035] T 4d801325b5a347a28e618b1f6eed879b P 234f2d4e0ee043eaad4521b5b83f5c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dca6cc23dc32473595146ec139d3b0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 37217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:06:32.575242 5261 mvcc.cc:205] Tried to move back new op lower bound from 7334743214373814272 to 7334743214334275584. Current Snapshot: MvccSnapshot[applied={T|T < 7334743214373814272}]
I20260929 19:06:32.576707 5263 mvcc.cc:205] Tried to move back new op lower bound from 7334743214373814272 to 7334743214334275584. Current Snapshot: MvccSnapshot[applied={T|T < 7334743214373814272 or (T in {7334743214373814272})}]
I20260929 19:06:32.579283 4775 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:32.579385 4775 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:06:32.580127 4775 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf3278be52684c85a21d0efb515e92db: Sending DeleteTablet for 3 replicas of tablet 4d801325b5a347a28e618b1f6eed879b
I20260929 19:06:32.580461 4900 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d801325b5a347a28e618b1f6eed879b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:32 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:34580
I20260929 19:06:32.580510 5034 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d801325b5a347a28e618b1f6eed879b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:32 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:54234
I20260929 19:06:32.580677 5168 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d801325b5a347a28e618b1f6eed879b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:06:32 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:49334
I20260929 19:06:32.580828 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4837
I20260929 19:06:32.582278 5273 tablet_replica.cc:330] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: stopping tablet replica
I20260929 19:06:32.582398 5273 raft_consensus.cc:2229] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:32.582489 5273 raft_consensus.cc:2258] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:32.582872 5272 tablet_replica.cc:330] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: stopping tablet replica
I20260929 19:06:32.582973 5273 ts_tablet_manager.cc:1905] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:32.582995 5272 raft_consensus.cc:2229] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:06:32.583094 5272 raft_consensus.cc:2258] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:06:32.583455 5272 ts_tablet_manager.cc:1905] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:06:32.584951 5273 ts_tablet_manager.cc:1918] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:32.585026 5273 log.cc:1196] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/4d801325b5a347a28e618b1f6eed879b
I20260929 19:06:32.584995 5272 ts_tablet_manager.cc:1918] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:06:32.585086 5272 log.cc:1196] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/4d801325b5a347a28e618b1f6eed879b
I20260929 19:06:32.585301 5273 ts_tablet_manager.cc:1939] T 4d801325b5a347a28e618b1f6eed879b P dca6cc23dc32473595146ec139d3b0d6: Deleting consensus metadata
I20260929 19:06:32.585368 5272 ts_tablet_manager.cc:1939] T 4d801325b5a347a28e618b1f6eed879b P 8390d0fbdf2b45878a929300db4c26b9: Deleting consensus metadata
I20260929 19:06:32.585752 4760 catalog_manager.cc:4841] TS 8390d0fbdf2b45878a929300db4c26b9 (127.26.78.194:44481): tablet 4d801325b5a347a28e618b1f6eed879b (table test-table [id=3dc5d249b4a74819b828cb72591025f6]) successfully deleted
I20260929 19:06:32.585772 4759 catalog_manager.cc:4841] TS dca6cc23dc32473595146ec139d3b0d6 (127.26.78.195:37217): tablet 4d801325b5a347a28e618b1f6eed879b (table test-table [id=3dc5d249b4a74819b828cb72591025f6]) successfully deleted
W20260929 19:06:32.586686 4762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:41199 (error 108)
I20260929 19:06:32.586887 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4971
I20260929 19:06:32.587589 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:06:32.587049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:41199 (local address 127.0.0.1:34600)
0929 19:06:32.587513 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0929 19:06:32.587549 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:41199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446}
W20260929 19:06:32.588445 4762 catalog_manager.cc:4584] TS 234f2d4e0ee043eaad4521b5b83f5c46 (127.26.78.193:41199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d801325b5a347a28e618b1f6eed879b: Network error: Client connection negotiation failed: client connection to 127.26.78.193:41199: connect: Connection refused (error 111)
I20260929 19:06:32.591789 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 5105
I20260929 19:06:32.596908 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 4743
2026-09-29T19:06:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1899 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[5277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[5277](info): set up 2 sockets
Sep 29 19:06:32 dist-test-slave-xz51 krb5kdc[5277](info): commencing operation
krb5kdc: starting...
W20260929 19:06:33.634613 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:06:33 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:06:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:06:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/rangeradmin_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/rangeradmin_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/rangerlookup_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/rangerlookup_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
I20260929 19:06:33.796332 26939 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZMrn7N/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/postgres -l logfile start
2026-09-29 19:06:36.329 UTC [5307] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 19:06:36.329 UTC [5307] LOG: listening on IPv4 address "127.26.78.212", port 52127
2026-09-29 19:06:36.333 UTC [5307] LOG: listening on Unix socket "/tmp/.s.PGSQL.52127"
2026-09-29 19:06:36.368 UTC [5310] LOG: database system was shut down at 2026-09-29 19:06:34 UTC
2026-09-29 19:06:36.372 UTC [5307] LOG: database system is ready to accept connections
I20260929 19:06:37.314270 26939 mini_postgres.cc:96] Postgres bound to 52127
2026-09-29 19:06:37.320 UTC [5318] FATAL: database "slave" does not exist
127.26.78.212:52127 - accepting connections
I20260929 19:06:37.321233 26939 mini_ranger.cc:161] Starting Ranger...
I20260929 19:06:37.336210 26939 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 19:06:37.832738 26939 mini_ranger.cc:87] Created ranger Postgres database
I20260929 19:06:37.832823 26939 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-admin
2026-09-29 19:06:38,112 [I] DB FLAVOR :POSTGRES
2026-09-29 19:06:38,113 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:06:38,113 [I] Checking connection..
2026-09-29 19:06:38,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 19:06:38,372 [I] Checking connection passed.
2026-09-29 19:06:38,372 [I] --------- Verifying version history table ---------
2026-09-29 19:06:38,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:06:38,615 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:06:38,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:06:38,863 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:06:38,863 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 19:06:38,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 19:06:39.097 UTC [5382] WARNING: there is no transaction in progress
2026-09-29 19:06:39,104 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 19:06:39,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:06:39,346 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 19:06:39,346 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:06:39,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 19:06:39,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 19:06:39,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xz51.c.gcp-upstream.internal','N') ;"
2026-09-29 19:06:40,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:06:40,367 [I] Table x_portal_user does not exist in database ranger
2026-09-29 19:06:40,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:06:40,631 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 19:06:40,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:06:40,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:06:41,153 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 19:06:41,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 19:06:42.110 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.127 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.145 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.161 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.181 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.202 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.270 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.277 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.292 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.304 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.315 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.330 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.341 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.348 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.355 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:42.362 UTC [5490] WARNING: there is no transaction in progress
2026-09-29 19:06:43,006 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 19:06:43,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 19:06:43,244 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 19:06:43,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:06:43,502 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 19:06:43,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:06:43,740 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 19:06:43,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:06:43,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:06:44,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xz51.c.gcp-upstream.internal';"
2026-09-29 19:06:44,450 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 19:06:44,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 19:06:44,698 [I] DB_PATCHES have already been applied
I20260929 19:06:44.704352 26939 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZMrn7N/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 19:06:44.704421 26939 mini_ranger.cc:193] Using host: 127.26.78.212
I20260929 19:06:44.707471 26939 mini_ranger.cc:227] Ranger admin URL: http://127.26.78.212:59507
Sep 29, 2026 7:06:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:06:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:06:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 7:06:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 7:06:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 7:06:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 7:06:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 19:06:45.815296 26939 mini_ranger.cc:160] Time spent starting Ranger: real 8.494s user 0.000s sys 0.009s
Sep 29, 2026 7:07:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 7:07:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 7:07:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 7:07:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 7:07:06 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 19:07:06.927986 26939 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/rangerkms_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/rangerkms_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.78.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.78.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 19:07:07 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708827, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 19:07:07.026947 26939 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 19:07:07.037626 26939 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 19:07:07.925102 26939 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 19:07:07.925189 26939 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms
I20260929 19:07:07.925211 26939 mini_ranger_kms.cc:227] Using postgres at 127.26.78.212:52127
2026-09-29 19:07:08,108 [I] DB FLAVOR :POSTGRES
2026-09-29 19:07:08,108 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:07:08,108 [I] Checking connection
2026-09-29 19:07:08,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 19:07:08,356 [I] connection success
2026-09-29 19:07:08,356 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 19:07:08,356 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 19:07:08,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 19:07:08,585 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 19:07:08,585 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:07:08,585 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 19:07:08,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:52127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 19:07:08,835 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 19:07:09.299975 26939 mini_ranger_kms.cc:310] Created kms service
I20260929 19:07:09.498562 26939 mini_ranger_kms.cc:326] Created kudu user
I20260929 19:07:09.544771 26939 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 19:07:09 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708827, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:07:09.953073 26939 mini_ranger_kms.cc:384] Added ranger policy
I20260929 19:07:09.953198 26939 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 19:07:09.953222 26939 mini_ranger_kms.cc:240] Using host: 127.26.78.212
I20260929 19:07:09.956467 26939 mini_ranger_kms.cc:276] Ranger KMS PID: 5670
I20260929 19:07:09.956602 26939 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.78.212:33761
Sep 29, 2026 7:07:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:07:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:07:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 7:07:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 7:07:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 7:07:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 19:07:12.190006 26939 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.163s user 0.000s sys 0.146s
I20260929 19:07:12.190115 26939 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 19:07:12.190196 26939 mini_ranger_kms.cc:401] 127.26.78.212:33761/kms/v1/keys
Sep 29 19:07:12 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790708832, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.78.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 19:07:12 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708832, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.78.212@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
Sep 29 19:07:13 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708832, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.78.212@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
Sep 29 19:07:15 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:07:15.471494 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44547
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44547
--ranger_config_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:33761/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:07:15.577688 5721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:07:15.577962 5721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:07:15.577987 5721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:07:15.579161 5721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:07:15.579200 5721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:07:15.579213 5721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:07:15.579226 5721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:07:15.579238 5721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:07:15.580776 5721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.78.212:33761/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44547
--ranger_config_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:07:15.581015 5721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:07:15.581242 5721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:07:15.587248 5727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:07:15.587256 5729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:07:15.587268 5726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:07:15.587524 5721 server_base.cc:1034] running on GCE node
I20260929 19:07:15.587761 5721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:07:15.588075 5721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:07:15.589248 5721 hybrid_clock.cc:648] HybridClock initialized: now 1790708835589238 us; error 30 us; skew 500 ppm
Sep 29 19:07:15 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708835, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:07:15.598584 5721 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:07:15.599120 5721 webserver.cc:460] Webserver started at http://127.26.78.254:41243/ using document root <none> and password file <none>
I20260929 19:07:15.599349 5721 fs_manager.cc:359] Metadata directory not provided
I20260929 19:07:15.599416 5721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:07:15.599563 5721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:07:15 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708835, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:07:15.645447 5721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "711c7422c74d461f83191606e8d6a6a1"
format_stamp: "Formatted at 2026-09-29 19:07:15 on dist-test-slave-xz51"
server_key: "3049cc3aeb4e44ef83d8e6071ea00f57"
server_key_iv: "5358b3c9e48cb8af1ab7edcc5843596b"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:15.645905 5721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "711c7422c74d461f83191606e8d6a6a1"
format_stamp: "Formatted at 2026-09-29 19:07:15 on dist-test-slave-xz51"
server_key: "3049cc3aeb4e44ef83d8e6071ea00f57"
server_key_iv: "5358b3c9e48cb8af1ab7edcc5843596b"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:15.647369 5721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:07:15.662533 5736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:15.662789 5721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:07:15.662915 5721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "711c7422c74d461f83191606e8d6a6a1"
format_stamp: "Formatted at 2026-09-29 19:07:15 on dist-test-slave-xz51"
server_key: "3049cc3aeb4e44ef83d8e6071ea00f57"
server_key_iv: "5358b3c9e48cb8af1ab7edcc5843596b"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:15.663018 5721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:07:15.676153 5721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:07:15.678161 5721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:07:15.678309 5721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:07:15.683753 5721 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:44547
I20260929 19:07:15.683774 5788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:44547 every 8 connection(s)
I20260929 19:07:15.684135 5721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:07:15.684816 5789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:07:15.687140 5789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1: Bootstrap starting.
I20260929 19:07:15.687714 5789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1: Neither blocks nor log segments found. Creating new log.
I20260929 19:07:15.688014 5789 log.cc:824] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:07:15.688601 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 5721
I20260929 19:07:15.688704 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:07:15.688881 5789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1: No bootstrap required, opened a new log
I20260929 19:07:15.688936 26939 external_mini_cluster.cc:1428] Setting key 1a63e610c1646ec5a9f2cc2d348a257d
I20260929 19:07:15.691924 5789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711c7422c74d461f83191606e8d6a6a1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44547 } }
I20260929 19:07:15.692086 5789 raft_consensus.cc:374] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:07:15.692147 5789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711c7422c74d461f83191606e8d6a6a1, State: Initialized, Role: FOLLOWER
I20260929 19:07:15.692283 5789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711c7422c74d461f83191606e8d6a6a1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44547 } }
I20260929 19:07:15.692402 5789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:07:15.692457 5789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:07:15.692510 5789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:07:15.693070 5789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711c7422c74d461f83191606e8d6a6a1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44547 } }
I20260929 19:07:15.693177 5789 leader_election.cc:302] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 711c7422c74d461f83191606e8d6a6a1; no voters:
I20260929 19:07:15.693387 5789 leader_election.cc:288] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:07:15.693434 5794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:07:15.693646 5794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [term 1 LEADER]: Becoming Leader. State: Replica: 711c7422c74d461f83191606e8d6a6a1, State: Running, Role: LEADER
I20260929 19:07:15.693773 5789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:07:15.693787 5794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711c7422c74d461f83191606e8d6a6a1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44547 } }
I20260929 19:07:15.695101 5794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "711c7422c74d461f83191606e8d6a6a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711c7422c74d461f83191606e8d6a6a1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44547 } } }
I20260929 19:07:15.695218 5794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:07:15.695364 5794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 711c7422c74d461f83191606e8d6a6a1. Latest consensus state: current_term: 1 leader_uuid: "711c7422c74d461f83191606e8d6a6a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711c7422c74d461f83191606e8d6a6a1" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44547 } } }
I20260929 19:07:15.695446 5794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:07:15.695544 5789 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 29 19:07:15 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:07:15.700946 5808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:07:15.702188 5808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:07:15.702941 5793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:15.689490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56490 (local address 127.26.78.254:44547)
0929 19:07:15.689688 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:15.689699 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:15.689811 (+ 112us) server_negotiation.cc:407] Connection header received
0929 19:07:15.690432 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:15.690444 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:15.690519 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:15.690677 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:07:15.691484 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:15.692954 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:15.694027 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:15.694637 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:15.694749 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:15.698463 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:07:15.698533 (+ 70us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:07:15.698533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:07:15.698643 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:07:15.701179 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:15.701592 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:15.701595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:15.701596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:15.701697 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:15.701941 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:15.701942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:15.701942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:15.702263 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:07:15.702431 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:15.702550 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 19:07:15.702705 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260929 19:07:15.706710 5812 mvcc.cc:205] Tried to move back new op lower bound from 7334743391052488704 to 7334743391006015488. Current Snapshot: MvccSnapshot[applied={T|T < 7334743391052488704}]
I20260929 19:07:15.708673 5808 catalog_manager.cc:1347] Generated new cluster ID: 4f5df205b6ec43a4b85a68247f62cbaa
I20260929 19:07:15.708720 5808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:07:15.722261 5808 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:07:15.723116 5808 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:07:15.736065 5808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1: Generated new TSK 0
I20260929 19:07:15.736370 5808 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:07:16.429024 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:33761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44547
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:07:16.529281 5834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:07:16.529486 5834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:07:16.529510 5834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:07:16.530710 5834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:07:16.530752 5834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:07:16.530778 5834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:07:16.532230 5834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.78.212:33761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:07:16.532474 5834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:07:16.532721 5834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:07:16.537727 5839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:07:16.537810 5834 server_base.cc:1034] running on GCE node
W20260929 19:07:16.537760 5842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:07:16.538003 5840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:07:16.538275 5834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:07:16.538548 5834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:07:16.539738 5834 hybrid_clock.cc:648] HybridClock initialized: now 1790708836539720 us; error 32 us; skew 500 ppm
Sep 29 19:07:16 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:07:16.542897 5834 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:07:16.543406 5834 webserver.cc:460] Webserver started at http://127.26.78.193:42051/ using document root <none> and password file <none>
I20260929 19:07:16.543690 5834 fs_manager.cc:359] Metadata directory not provided
I20260929 19:07:16.543751 5834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:07:16.543874 5834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:07:16 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:07:16.558102 5834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "ca49f8f9fa1f4137a026c202a24aa45c"
format_stamp: "Formatted at 2026-09-29 19:07:16 on dist-test-slave-xz51"
server_key: "765a856898283a55a4886b43e87eff6d"
server_key_iv: "e35c4cca163d0291150e95806aeb0142"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:16.558602 5834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "ca49f8f9fa1f4137a026c202a24aa45c"
format_stamp: "Formatted at 2026-09-29 19:07:16 on dist-test-slave-xz51"
server_key: "765a856898283a55a4886b43e87eff6d"
server_key_iv: "e35c4cca163d0291150e95806aeb0142"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:16.561769 5834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260929 19:07:16.580799 5851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:16.581085 5834 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260929 19:07:16.581190 5834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "ca49f8f9fa1f4137a026c202a24aa45c"
format_stamp: "Formatted at 2026-09-29 19:07:16 on dist-test-slave-xz51"
server_key: "765a856898283a55a4886b43e87eff6d"
server_key_iv: "e35c4cca163d0291150e95806aeb0142"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:16.581295 5834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:07:16.605337 5834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:07:16.607980 5834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:07:16.608201 5834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:07:16.608533 5834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:07:16.608938 5834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:07:16.609036 5834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:16.609138 5834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:07:16.609222 5834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:16.616752 5834 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:42071
I20260929 19:07:16.616845 5964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:42071 every 8 connection(s)
I20260929 19:07:16.617419 5834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:07:16 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:07:16.626063 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 5834
I20260929 19:07:16.626157 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:07:16.626370 26939 external_mini_cluster.cc:1428] Setting key 5c70af42b202107f8ea24169c254d547
I20260929 19:07:16.630332 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:16.618937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:41497 (local address 127.26.78.254:44547)
0929 19:07:16.619107 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:16.619110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:16.619128 (+ 18us) server_negotiation.cc:407] Connection header received
0929 19:07:16.619830 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:16.619832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:16.619848 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:16.619945 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:07:16.620714 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.621532 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.623476 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.623793 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.623882 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:16.626888 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:07:16.626889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:07:16.626889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:07:16.626897 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:07:16.628711 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:16.629194 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:16.629195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:16.629196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:16.629265 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:16.629572 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:16.629573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:16.629573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:16.629715 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:07:16.629786 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:16.630141 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 19:07:16.630230 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260929 19:07:16.630504 5967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:16.618628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44547 (local address 127.26.78.193:41497)
0929 19:07:16.618831 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:16.618886 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:16.619697 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:16.620050 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:16.620055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:16.620318 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:07:16.620597 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:16.620605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.621643 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.621644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:16.623374 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:16.623377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.623900 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.623901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:16.624049 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:16.624681 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:07:16.624691 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:07:16.626791 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:07:16.628799 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:07:16.628806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:07:16.628809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:07:16.629113 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:07:16.629347 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:07:16.629348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:07:16.629349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:07:16.629491 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:07:16.629796 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:07:16.629803 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:07:16.629984 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 19:07:16.630106 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 19:07:16.630271 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260929 19:07:16.631141 5965 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44547
I20260929 19:07:16.631249 5965 heartbeater.cc:464] Registering TS with master...
I20260929 19:07:16.631448 5965 heartbeater.cc:511] Master 127.26.78.254:44547 requested a full tablet report, sending...
I20260929 19:07:16.632051 5753 ts_manager.cc:194] Registered new tserver with Master: ca49f8f9fa1f4137a026c202a24aa45c (127.26.78.193:42071)
I20260929 19:07:16.632952 5753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:41497
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:07:16.664973 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:33761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44547
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:07:16 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 19:07:16.787520 5971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:07:16.787797 5971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:07:16.787833 5971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:07:16.789530 5971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:07:16.789660 5971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:07:16.789747 5971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:07:16.792047 5971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.78.212:33761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:07:16.792518 5971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:07:16.792837 5971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:07:16.801128 5978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:07:16.801440 5979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:07:16.801574 5981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:07:16.802676 5971 server_base.cc:1034] running on GCE node
I20260929 19:07:16.802922 5971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:07:16.803205 5971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:07:16.804427 5971 hybrid_clock.cc:648] HybridClock initialized: now 1790708836804398 us; error 41 us; skew 500 ppm
Sep 29 19:07:16 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:07:16.808310 5971 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:07:16.808813 5971 webserver.cc:460] Webserver started at http://127.26.78.194:36379/ using document root <none> and password file <none>
I20260929 19:07:16.809062 5971 fs_manager.cc:359] Metadata directory not provided
I20260929 19:07:16.809118 5971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:07:16.809242 5971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:07:16 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:07:16.824129 5971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "5c34e3722d6145669f69cf0876bb9005"
format_stamp: "Formatted at 2026-09-29 19:07:16 on dist-test-slave-xz51"
server_key: "c8f40133ce9f640166243fc049dc50ac"
server_key_iv: "f200a2a60c2cd08c3697111f1cc9e891"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:16.824709 5971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "5c34e3722d6145669f69cf0876bb9005"
format_stamp: "Formatted at 2026-09-29 19:07:16 on dist-test-slave-xz51"
server_key: "c8f40133ce9f640166243fc049dc50ac"
server_key_iv: "f200a2a60c2cd08c3697111f1cc9e891"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:16.826939 5971 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260929 19:07:16.839444 5989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:16.839757 5971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:07:16.839915 5971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "5c34e3722d6145669f69cf0876bb9005"
format_stamp: "Formatted at 2026-09-29 19:07:16 on dist-test-slave-xz51"
server_key: "c8f40133ce9f640166243fc049dc50ac"
server_key_iv: "f200a2a60c2cd08c3697111f1cc9e891"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:16.840099 5971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:07:16.855624 5971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:07:16.858136 5971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:07:16.858345 5971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:07:16.858655 5971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:07:16.859092 5971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:07:16.859190 5971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:16.859265 5971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:07:16.859334 5971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:16.868598 5971 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:38347
I20260929 19:07:16.869125 5971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:07:16.869716 6102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:38347 every 8 connection(s)
I20260929 19:07:16.873961 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 5971
I20260929 19:07:16.874070 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:07:16.874266 26939 external_mini_cluster.cc:1428] Setting key e2de2b19e4b54e2b4c0e15ea63f67a86
Sep 29 19:07:16 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:07:16.885288 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:16.870772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:42429 (local address 127.26.78.254:44547)
0929 19:07:16.870820 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:16.870823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:16.871183 (+ 360us) server_negotiation.cc:407] Connection header received
0929 19:07:16.871971 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:16.871973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:16.871990 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:16.872024 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:07:16.872893 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.873703 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.878379 (+ 4676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.878688 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.878739 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:16.881482 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:07:16.881484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:07:16.881485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:07:16.881493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:07:16.883312 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:16.884095 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:16.884097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:16.884097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:16.884167 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:16.884503 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:16.884504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:16.884505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:16.884651 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:07:16.884728 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:16.885089 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 19:07:16.885185 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:07:16.885546 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:16.870517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44547 (local address 127.26.78.194:42429)
0929 19:07:16.871046 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:16.871092 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:16.871790 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:16.872219 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:16.872224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:16.872482 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:07:16.872771 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:16.872779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.876851 (+ 4072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.876852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:16.878247 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:16.878250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:16.878845 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:16.878846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:16.878999 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:16.879645 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:07:16.879656 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:07:16.881349 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:07:16.883674 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:07:16.883682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:07:16.883684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:07:16.883991 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:07:16.884254 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:07:16.884255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:07:16.884256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:07:16.884419 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:07:16.884736 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:07:16.884744 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:07:16.884923 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 19:07:16.885051 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 19:07:16.885222 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":76,"threads_started":1}
I20260929 19:07:16.886224 6103 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44547
I20260929 19:07:16.886337 6103 heartbeater.cc:464] Registering TS with master...
I20260929 19:07:16.886545 6103 heartbeater.cc:511] Master 127.26.78.254:44547 requested a full tablet report, sending...
I20260929 19:07:16.886914 5753 ts_manager.cc:194] Registered new tserver with Master: 5c34e3722d6145669f69cf0876bb9005 (127.26.78.194:38347)
I20260929 19:07:16.887410 5753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:42429
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:07:16.913756 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:33761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44547
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:07:17.022696 6108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:07:17.023025 6108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:07:17.023069 6108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:07:17.024274 6108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:07:17.024353 6108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:07:17.024398 6108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:07:17.026118 6108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.78.212:33761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:07:17.026365 6108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:07:17.026625 6108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:07:17.032034 6114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:07:17.032034 6113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:07:17.032298 6108 server_base.cc:1034] running on GCE node
W20260929 19:07:17.032052 6116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:07:17.032670 6108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:07:17.032934 6108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:07:17.034094 6108 hybrid_clock.cc:648] HybridClock initialized: now 1790708837034072 us; error 41 us; skew 500 ppm
Sep 29 19:07:17 dist-test-slave-xz51 krb5kdc[5277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708837, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:07:17.036691 6108 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:07:17.037124 6108 webserver.cc:460] Webserver started at http://127.26.78.195:44669/ using document root <none> and password file <none>
I20260929 19:07:17.037367 6108 fs_manager.cc:359] Metadata directory not provided
I20260929 19:07:17.037439 6108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:07:17.037590 6108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:07:17 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708837, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:07:17.050701 6108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "76143976f2c34323bba222b85ac0d0d6"
format_stamp: "Formatted at 2026-09-29 19:07:17 on dist-test-slave-xz51"
server_key: "d356e20de5c4f836d6eb952717f359bd"
server_key_iv: "cb8e778adf5c104ba1e8919f2ec46dbc"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:17.051180 6108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "76143976f2c34323bba222b85ac0d0d6"
format_stamp: "Formatted at 2026-09-29 19:07:17 on dist-test-slave-xz51"
server_key: "d356e20de5c4f836d6eb952717f359bd"
server_key_iv: "cb8e778adf5c104ba1e8919f2ec46dbc"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:17.053094 6108 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260929 19:07:17.078002 6123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:17.078298 6108 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.000s sys 0.000s
I20260929 19:07:17.078426 6108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "76143976f2c34323bba222b85ac0d0d6"
format_stamp: "Formatted at 2026-09-29 19:07:17 on dist-test-slave-xz51"
server_key: "d356e20de5c4f836d6eb952717f359bd"
server_key_iv: "cb8e778adf5c104ba1e8919f2ec46dbc"
server_key_version: "kuduclusterkey@0"
I20260929 19:07:17.078537 6108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:07:17.091843 6108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:07:17.094661 6108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:07:17.094837 6108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:07:17.095085 6108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:07:17.095480 6108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:07:17.095548 6108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:17.095615 6108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:07:17.095644 6108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:07:17.103266 6108 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:39509
I20260929 19:07:17.103796 6108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:07:17.105791 6236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:39509 every 8 connection(s)
I20260929 19:07:17.106601 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 6108
I20260929 19:07:17.106712 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:07:17.106909 26939 external_mini_cluster.cc:1428] Setting key f97cc827cfeed21cfcc1bf0d3dd97397
Sep 29 19:07:17 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708837, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:07:17.115751 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.104966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:46287 (local address 127.26.78.254:44547)
0929 19:07:17.105020 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.105023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.105250 (+ 227us) server_negotiation.cc:407] Connection header received
0929 19:07:17.106368 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.106370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.106384 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.106443 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:07:17.107918 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.108689 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.109703 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.109994 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.110071 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.112388 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:07:17.112389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:07:17.112390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:07:17.112399 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:07:17.114174 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:17.114663 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:17.114664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:17.114665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:17.114730 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:17.115072 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:17.115073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:17.115074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:17.115227 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:07:17.115264 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.115571 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.115657 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:07:17.116101 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.104633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44547 (local address 127.26.78.195:46287)
0929 19:07:17.105102 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.105155 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:17.106234 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:17.107168 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:17.107173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:17.107508 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:07:17.107820 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.107826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.108790 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.108791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.109597 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.109599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.110137 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.110138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.110278 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.110865 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:07:17.110882 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:07:17.112276 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:07:17.114274 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:07:17.114280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:07:17.114282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:07:17.114579 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:07:17.114808 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:07:17.114809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:07:17.114810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:07:17.114982 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:07:17.115304 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:07:17.115309 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:07:17.115437 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 19:07:17.115553 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 19:07:17.115700 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":82,"threads_started":1}
I20260929 19:07:17.116750 6237 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44547
I20260929 19:07:17.116843 6237 heartbeater.cc:464] Registering TS with master...
I20260929 19:07:17.117039 6237 heartbeater.cc:511] Master 127.26.78.254:44547 requested a full tablet report, sending...
I20260929 19:07:17.117486 5753 ts_manager.cc:194] Registered new tserver with Master: 76143976f2c34323bba222b85ac0d0d6 (127.26.78.195:39509)
I20260929 19:07:17.118022 5753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:46287
I20260929 19:07:17.119208 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:07:17.132443 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.121225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.26.78.254:44547)
0929 19:07:17.121271 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.121273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.121547 (+ 274us) server_negotiation.cc:407] Connection header received
0929 19:07:17.121651 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.121652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.121663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.121695 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:07:17.122566 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.123397 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.124508 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.124895 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.124961 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.128861 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:07:17.128862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:07:17.128862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:07:17.128871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:07:17.131156 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:17.131591 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:17.131592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:17.131593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:17.131651 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:07:17.131852 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:07:17.131853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:07:17.131854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:07:17.131996 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:07:17.132067 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.132286 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.132367 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:07:17.134641 5751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:07:17.136567 5751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:07:17.146615 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.139008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:39509 (local address 127.0.0.1:39250)
0929 19:07:17.142312 (+ 3304us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.142321 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:17.142363 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:17.142625 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:17.142626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:17.142628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:07:17.142668 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.142669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.143910 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.143912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.145231 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.145233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.146299 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.146300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.146384 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.146401 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 19:07:17.146476 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:07:17.146528 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3292,"thread_start_us":79,"threads_started":1}
I20260929 19:07:17.147119 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.139424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.26.78.195:39509)
0929 19:07:17.139569 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.139571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.142423 (+ 2852us) server_negotiation.cc:407] Connection header received
0929 19:07:17.142465 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.142467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.142483 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.142551 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:07:17.142746 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.143792 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.145331 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.146115 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.146189 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.146264 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.146997 (+ 733us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.147037 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260929 19:07:17.147994 6171 tablet_service.cc:1467] Processing CreateTablet for tablet 9a2d631fec5944a785d6325de2449544 (DEFAULT_TABLE table=test-table [id=65ecd19532384e9e8446b26e80154b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:07:17.148382 6171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a2d631fec5944a785d6325de2449544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:07:17.151257 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.138957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42071 (local address 127.0.0.1:47796)
0929 19:07:17.141970 (+ 3013us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.142019 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:17.142091 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:17.147638 (+ 5547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:17.147640 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:17.147641 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:07:17.147681 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.147682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.149027 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.149028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.150101 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.150103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.151031 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.151032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.151093 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.151098 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:07:17.151129 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 19:07:17.151170 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2978,"thread_start_us":88,"threads_started":1}
I20260929 19:07:17.151681 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.139207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.26.78.193:42071)
0929 19:07:17.147445 (+ 8238us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.147450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.147463 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:07:17.147500 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.147502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.147520 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.147760 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:07:17.147829 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.148921 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.150197 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.150929 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.151453 (+ 524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.151530 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.151558 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.151602 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8197,"thread_start_us":61,"threads_started":1}
I20260929 19:07:17.152056 6255 tablet_bootstrap.cc:492] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Bootstrap starting.
I20260929 19:07:17.152661 5898 tablet_service.cc:1467] Processing CreateTablet for tablet 9a2d631fec5944a785d6325de2449544 (DEFAULT_TABLE table=test-table [id=65ecd19532384e9e8446b26e80154b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:07:17.153018 5898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a2d631fec5944a785d6325de2449544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:07:17.153170 6255 tablet_bootstrap.cc:654] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Neither blocks nor log segments found. Creating new log.
I20260929 19:07:17.153239 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.139240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.26.78.194:38347)
0929 19:07:17.139425 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.139429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.142245 (+ 2816us) server_negotiation.cc:407] Connection header received
0929 19:07:17.142302 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.142305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.142324 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.142418 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:07:17.142681 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.145119 (+ 2438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.149534 (+ 4415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.152259 (+ 2725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.152331 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.152435 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.153099 (+ 664us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.153154 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260929 19:07:17.152755 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.138974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38347 (local address 127.0.0.1:44830)
0929 19:07:17.142174 (+ 3200us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.142181 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:17.142240 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:17.142517 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:17.142520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:17.142522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:07:17.142598 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.142600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.145204 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.145205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.149415 (+ 4210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.149417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.152477 (+ 3060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.152478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.152549 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.152554 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:07:17.152611 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 19:07:17.152662 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3188,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260929 19:07:17.153553 6255 log.cc:824] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:07:17.154703 6255 tablet_bootstrap.cc:492] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: No bootstrap required, opened a new log
I20260929 19:07:17.154750 6255 ts_tablet_manager.cc:1397] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260929 19:07:17.156589 6255 raft_consensus.cc:348] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.156744 6255 raft_consensus.cc:374] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:07:17.156773 6255 raft_consensus.cc:729] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76143976f2c34323bba222b85ac0d0d6, State: Initialized, Role: FOLLOWER
I20260929 19:07:17.156927 6255 consensus_queue.cc:260] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.157231 6237 heartbeater.cc:503] Master 127.26.78.254:44547 was elected leader, sending a full tablet report...
I20260929 19:07:17.157470 6255 ts_tablet_manager.cc:1428] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:07:17.157653 6257 tablet_bootstrap.cc:492] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c: Bootstrap starting.
I20260929 19:07:17.157581 6037 tablet_service.cc:1467] Processing CreateTablet for tablet 9a2d631fec5944a785d6325de2449544 (DEFAULT_TABLE table=test-table [id=65ecd19532384e9e8446b26e80154b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:07:17.157979 6037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a2d631fec5944a785d6325de2449544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:07:17.158543 6257 tablet_bootstrap.cc:654] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c: Neither blocks nor log segments found. Creating new log.
I20260929 19:07:17.158856 6257 log.cc:824] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c: Log is configured to *not* fsync() on all Append() calls
I20260929 19:07:17.159556 6257 tablet_bootstrap.cc:492] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c: No bootstrap required, opened a new log
I20260929 19:07:17.159600 6257 ts_tablet_manager.cc:1397] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:07:17.162134 6257 raft_consensus.cc:348] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.162377 6257 raft_consensus.cc:374] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:07:17.162472 6257 raft_consensus.cc:729] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca49f8f9fa1f4137a026c202a24aa45c, State: Initialized, Role: FOLLOWER
I20260929 19:07:17.162662 6257 consensus_queue.cc:260] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.162999 6257 ts_tablet_manager.cc:1428] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260929 19:07:17.163160 5965 heartbeater.cc:503] Master 127.26.78.254:44547 was elected leader, sending a full tablet report...
I20260929 19:07:17.166003 6260 tablet_bootstrap.cc:492] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: Bootstrap starting.
I20260929 19:07:17.167484 6260 tablet_bootstrap.cc:654] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: Neither blocks nor log segments found. Creating new log.
I20260929 19:07:17.167867 6260 log.cc:824] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: Log is configured to *not* fsync() on all Append() calls
I20260929 19:07:17.168602 6260 tablet_bootstrap.cc:492] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: No bootstrap required, opened a new log
I20260929 19:07:17.168686 6260 ts_tablet_manager.cc:1397] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:07:17.170557 6260 raft_consensus.cc:348] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.170785 6260 raft_consensus.cc:374] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:07:17.170887 6260 raft_consensus.cc:729] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c34e3722d6145669f69cf0876bb9005, State: Initialized, Role: FOLLOWER
I20260929 19:07:17.171092 6260 consensus_queue.cc:260] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.171435 6260 ts_tablet_manager.cc:1428] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:07:17.171607 6103 heartbeater.cc:503] Master 127.26.78.254:44547 was elected leader, sending a full tablet report...
Sep 29 19:07:17 dist-test-slave-xz51 krb5kdc[5277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708836, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:07:17.253504 6258 raft_consensus.cc:482] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:07:17.253654 6258 raft_consensus.cc:504] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.254323 6258 leader_election.cc:288] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca49f8f9fa1f4137a026c202a24aa45c (127.26.78.193:42071), 5c34e3722d6145669f69cf0876bb9005 (127.26.78.194:38347)
I20260929 19:07:17.258831 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.254791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:40269 (local address 127.26.78.193:42071)
0929 19:07:17.254911 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.254913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.254921 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:07:17.254947 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.254948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.254959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.255013 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:07:17.255734 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.256394 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.257807 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.258434 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.258482 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.258491 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.258702 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.258757 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260929 19:07:17.258838 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.254513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38347 (local address 127.26.78.195:53855)
0929 19:07:17.255061 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.255070 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:17.255110 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:17.255267 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:17.255268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:17.255269 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:07:17.255340 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.255342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.256442 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.256443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.257885 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.257887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.258606 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.258607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.258687 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.258689 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:07:17.258733 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 19:07:17.258766 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":210,"thread_start_us":132,"threads_started":1}
I20260929 19:07:17.259261 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.254860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:53855 (local address 127.26.78.194:38347)
0929 19:07:17.254904 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.254906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.255101 (+ 195us) server_negotiation.cc:407] Connection header received
0929 19:07:17.255161 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.255162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.255173 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.255219 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:07:17.255425 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.256180 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.257977 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.258512 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.258573 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.258581 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.259144 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.259190 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:07:17.259572 6054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2d631fec5944a785d6325de2449544" candidate_uuid: "76143976f2c34323bba222b85ac0d0d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c34e3722d6145669f69cf0876bb9005" is_pre_election: true
I20260929 19:07:17.259771 6054 raft_consensus.cc:2454] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76143976f2c34323bba222b85ac0d0d6 in term 0.
I20260929 19:07:17.258850 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.254456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42071 (local address 127.26.78.195:40269)
0929 19:07:17.254706 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.254719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:07:17.254776 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:07:17.255579 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:07:17.255580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:07:17.255582 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:07:17.255652 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.255653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.256480 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.256481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.257714 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:07:17.257717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.258553 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.258554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:07:17.258625 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.258630 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:07:17.258687 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 19:07:17.258740 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":7}
I20260929 19:07:17.260036 6124 leader_election.cc:302] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c34e3722d6145669f69cf0876bb9005, 76143976f2c34323bba222b85ac0d0d6; no voters:
I20260929 19:07:17.260187 6258 raft_consensus.cc:2781] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:07:17.260243 6258 raft_consensus.cc:482] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:07:17.260270 6258 raft_consensus.cc:3037] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:07:17.261173 6258 raft_consensus.cc:504] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.261319 6258 leader_election.cc:288] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [CANDIDATE]: Term 1 election: Requested vote from peers ca49f8f9fa1f4137a026c202a24aa45c (127.26.78.193:42071), 5c34e3722d6145669f69cf0876bb9005 (127.26.78.194:38347)
I20260929 19:07:17.261497 5914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2d631fec5944a785d6325de2449544" candidate_uuid: "76143976f2c34323bba222b85ac0d0d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" is_pre_election: true
I20260929 19:07:17.261662 5914 raft_consensus.cc:2454] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76143976f2c34323bba222b85ac0d0d6 in term 0.
I20260929 19:07:17.261507 5915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2d631fec5944a785d6325de2449544" candidate_uuid: "76143976f2c34323bba222b85ac0d0d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca49f8f9fa1f4137a026c202a24aa45c"
I20260929 19:07:17.261801 5915 raft_consensus.cc:3037] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:07:17.262691 5915 raft_consensus.cc:2454] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76143976f2c34323bba222b85ac0d0d6 in term 1.
I20260929 19:07:17.262919 6127 leader_election.cc:302] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76143976f2c34323bba222b85ac0d0d6, ca49f8f9fa1f4137a026c202a24aa45c; no voters:
I20260929 19:07:17.262930 6054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2d631fec5944a785d6325de2449544" candidate_uuid: "76143976f2c34323bba222b85ac0d0d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c34e3722d6145669f69cf0876bb9005"
I20260929 19:07:17.263010 6054 raft_consensus.cc:3037] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:07:17.263854 6054 raft_consensus.cc:2454] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76143976f2c34323bba222b85ac0d0d6 in term 1.
I20260929 19:07:17.264232 6258 raft_consensus.cc:2781] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:07:17.264411 6258 raft_consensus.cc:686] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 1 LEADER]: Becoming Leader. State: Replica: 76143976f2c34323bba222b85ac0d0d6, State: Running, Role: LEADER
I20260929 19:07:17.264506 6258 consensus_queue.cc:237] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } }
I20260929 19:07:17.265386 5751 catalog_manager.cc:5495] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76143976f2c34323bba222b85ac0d0d6 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "76143976f2c34323bba222b85ac0d0d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76143976f2c34323bba222b85ac0d0d6" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 } health_report { overall_health: UNKNOWN } } }
I20260929 19:07:17.289391 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.285963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39260 (local address 127.26.78.195:39509)
0929 19:07:17.286007 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:07:17.286009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:07:17.286050 (+ 41us) server_negotiation.cc:407] Connection header received
0929 19:07:17.286149 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:07:17.286150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:07:17.286172 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:07:17.286293 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:07:17.286463 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.287214 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.288373 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:07:17.288650 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:07:17.288707 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:07:17.288886 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:07:17.289016 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:07:17.289085 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:07:17.289284 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 19:07:17.289302 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:07:17.292053 6054 raft_consensus.cc:1261] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 1 FOLLOWER]: Refusing update from remote peer 76143976f2c34323bba222b85ac0d0d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:07:17.292500 6265 consensus_queue.cc:1035] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c34e3722d6145669f69cf0876bb9005" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:07:17.294327 5915 raft_consensus.cc:1261] T 9a2d631fec5944a785d6325de2449544 P ca49f8f9fa1f4137a026c202a24aa45c [term 1 FOLLOWER]: Refusing update from remote peer 76143976f2c34323bba222b85ac0d0d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:07:17.294746 6265 consensus_queue.cc:1035] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca49f8f9fa1f4137a026c202a24aa45c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:07:17.296705 6268 mvcc.cc:205] Tried to move back new op lower bound from 7334743397545070592 to 7334743397436018688. Current Snapshot: MvccSnapshot[applied={T|T < 7334743397545070592}]
I20260929 19:07:17.297719 6269 mvcc.cc:205] Tried to move back new op lower bound from 7334743397545070592 to 7334743397436018688. Current Snapshot: MvccSnapshot[applied={T|T < 7334743397545070592}]
I20260929 19:07:17.304342 5751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:07:17.304455 5751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:07:17.305241 5751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 711c7422c74d461f83191606e8d6a6a1: Sending DeleteTablet for 3 replicas of tablet 9a2d631fec5944a785d6325de2449544
I20260929 19:07:17.305720 6171 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a2d631fec5944a785d6325de2449544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:07:17 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:39250
I20260929 19:07:17.305786 5898 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a2d631fec5944a785d6325de2449544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:07:17 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:47796
I20260929 19:07:17.306027 6037 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a2d631fec5944a785d6325de2449544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:07:17 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:44830
I20260929 19:07:17.306536 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 5834
W20260929 19:07:17.317659 6127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:42071 (error 108)
I20260929 19:07:17.318126 6278 tablet_replica.cc:330] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: stopping tablet replica
I20260929 19:07:17.318367 6278 raft_consensus.cc:2229] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:07:17.318574 6278 raft_consensus.cc:2258] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:07:17.318789 6276 tablet_replica.cc:330] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: stopping tablet replica
I20260929 19:07:17.319263 6278 ts_tablet_manager.cc:1905] T 9a2d631fec5944a785d6325de2449544 P 5c34e3722d6145669f69cf0876bb9005: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:07:17.321053 6276 raft_consensus.cc:2229] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:07:17.321230 6276 raft_consensus.cc:2258] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:07:17.321743 6276 ts_tablet_manager.cc:1905] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:07:17.323565 5740 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:42071 (error 108)
I20260929 19:07:17.323724 6276 ts_tablet_manager.cc:1918] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:07:17.323786 6276 log.cc:1196] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/9a2d631fec5944a785d6325de2449544
I20260929 19:07:17.324072 6276 ts_tablet_manager.cc:1939] T 9a2d631fec5944a785d6325de2449544 P 76143976f2c34323bba222b85ac0d0d6: Deleting consensus metadata
I20260929 19:07:17.324275 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 5971
I20260929 19:07:17.325124 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.324986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42071 (local address 127.26.78.195:41015)
0929 19:07:17.325031 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.325081 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:42071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260929 19:07:17.325598 5739 catalog_manager.cc:4841] TS 76143976f2c34323bba222b85ac0d0d6 (127.26.78.195:39509): tablet 9a2d631fec5944a785d6325de2449544 (table test-table [id=65ecd19532384e9e8446b26e80154b8d]) successfully deleted
I20260929 19:07:17.338280 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.338110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42071 (local address 127.0.0.1:47810)
0929 19:07:17.338162 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.338223 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:42071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 19:07:17.338376 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:07:17.338284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38347 (local address 127.0.0.1:44840)
0929 19:07:17.338324 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 19:07:17.338353 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.194:38347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260929 19:07:17.338450 5740 catalog_manager.cc:4584] TS ca49f8f9fa1f4137a026c202a24aa45c (127.26.78.193:42071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a2d631fec5944a785d6325de2449544: Network error: Client connection negotiation failed: client connection to 127.26.78.193:42071: connect: Connection refused (error 111)
I20260929 19:07:17.339037 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 6108
I20260929 19:07:17.344074 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 5721
I20260929 19:07:17.351328 26939 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 19:07:17.695919 26939 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 19:07:17.696017 26939 mini_ranger.cc:66] Stopping Ranger...
I20260929 19:07:18.040340 26939 mini_ranger.cc:68] Stopped Ranger
2026-09-29 19:07:18.040 UTC [5307] LOG: received smart shutdown request
2026-09-29 19:07:18.044 UTC [5307] LOG: background worker "logical replication launcher" (PID 5316) exited with exit code 1
2026-09-29 19:07:18.044 UTC [5311] LOG: shutting down
2026-09-29 19:07:18.226 UTC [5307] LOG: database system is shut down
2026-09-29T19:07:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45739 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:07:18 dist-test-slave-xz51 krb5kdc[6289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:07:18 dist-test-slave-xz51 krb5kdc[6289](info): set up 2 sockets
Sep 29 19:07:18 dist-test-slave-xz51 krb5kdc[6289](info): commencing operation
krb5kdc: starting...
W20260929 19:07:19.365751 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:07:19 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:07:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:07:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/rangeradmin_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/rangeradmin_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/rangerlookup_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/rangerlookup_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
I20260929 19:07:19.532475 26939 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZMrn7N/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/postgres -l logfile start
2026-09-29 19:07:22.026 UTC [6319] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 19:07:22.026 UTC [6319] LOG: listening on IPv4 address "127.26.78.212", port 39435
2026-09-29 19:07:22.030 UTC [6319] LOG: listening on Unix socket "/tmp/.s.PGSQL.39435"
2026-09-29 19:07:22.052 UTC [6322] LOG: database system was shut down at 2026-09-29 19:07:20 UTC
2026-09-29 19:07:22.055 UTC [6319] LOG: database system is ready to accept connections
I20260929 19:07:23.005699 26939 mini_postgres.cc:96] Postgres bound to 39435
2026-09-29 19:07:23.011 UTC [6330] FATAL: database "slave" does not exist
127.26.78.212:39435 - accepting connections
I20260929 19:07:23.012145 26939 mini_ranger.cc:161] Starting Ranger...
I20260929 19:07:23.026561 26939 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 19:07:23.506326 26939 mini_ranger.cc:87] Created ranger Postgres database
I20260929 19:07:23.506409 26939 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-admin
2026-09-29 19:07:23,784 [I] DB FLAVOR :POSTGRES
2026-09-29 19:07:23,785 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:07:23,785 [I] Checking connection..
2026-09-29 19:07:23,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 19:07:24,031 [I] Checking connection passed.
2026-09-29 19:07:24,031 [I] --------- Verifying version history table ---------
2026-09-29 19:07:24,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:07:24,268 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:07:24,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:07:24,503 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:07:24,503 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 19:07:24,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 19:07:24.739 UTC [6394] WARNING: there is no transaction in progress
2026-09-29 19:07:24,745 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 19:07:24,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:07:24,996 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 19:07:24,997 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:07:24,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 19:07:25,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 19:07:25,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xz51.c.gcp-upstream.internal','N') ;"
2026-09-29 19:07:25,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:07:25,920 [I] Table x_portal_user does not exist in database ranger
2026-09-29 19:07:25,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:07:26,176 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 19:07:26,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:07:26,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:07:26,647 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 19:07:26,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 19:07:27.607 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.624 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.641 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.657 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.674 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.700 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.763 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.771 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.786 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.797 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.807 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.822 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.833 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.839 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.847 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:27.854 UTC [6502] WARNING: there is no transaction in progress
2026-09-29 19:07:28,499 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 19:07:28,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 19:07:28,743 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 19:07:28,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:07:28,991 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 19:07:28,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:07:29,234 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 19:07:29,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:07:29,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:07:29,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xz51.c.gcp-upstream.internal';"
2026-09-29 19:07:29,970 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 19:07:29,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 19:07:30,233 [I] DB_PATCHES have already been applied
I20260929 19:07:30.239096 26939 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZMrn7N/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 19:07:30.239169 26939 mini_ranger.cc:193] Using host: 127.26.78.212
I20260929 19:07:30.242318 26939 mini_ranger.cc:227] Ranger admin URL: http://127.26.78.212:39437
Sep 29, 2026 7:07:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:07:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:07:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 7:07:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 7:07:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 7:07:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 7:07:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 19:07:31.349205 26939 mini_ranger.cc:160] Time spent starting Ranger: real 8.337s user 0.000s sys 0.008s
Sep 29, 2026 7:07:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 7:07:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 7:07:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 7:07:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 7:07:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 19:07:53.330746 26939 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.78.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.78.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/rangerkms_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.78.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/rangerkms_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.78.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.78.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.78.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/HTTP_127.26.78.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 19:07:53 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708873, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 19:07:53.415930 26939 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 19:07:53.426541 26939 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 19:07:54.278887 26939 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 19:07:54.278985 26939 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms
I20260929 19:07:54.279003 26939 mini_ranger_kms.cc:227] Using postgres at 127.26.78.212:39435
2026-09-29 19:07:54,444 [I] DB FLAVOR :POSTGRES
2026-09-29 19:07:54,445 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:07:54,445 [I] Checking connection
2026-09-29 19:07:54,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 19:07:54,687 [I] connection success
2026-09-29 19:07:54,688 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 19:07:54,688 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 19:07:54,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 19:07:54,924 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 19:07:54,925 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:07:54,925 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 19:07:54,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.78.212:39435/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 19:07:55,217 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 19:07:55.698522 26939 mini_ranger_kms.cc:310] Created kms service
I20260929 19:07:55.903725 26939 mini_ranger_kms.cc:326] Created kudu user
I20260929 19:07:55.950105 26939 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 19:07:55 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708873, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:07:56.337253 26939 mini_ranger_kms.cc:384] Added ranger policy
I20260929 19:07:56.337473 26939 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZMrn7N/build/release/bin/postgresql.jar:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZMrn7N/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 19:07:56.337498 26939 mini_ranger_kms.cc:240] Using host: 127.26.78.212
I20260929 19:07:56.341187 26939 mini_ranger_kms.cc:276] Ranger KMS PID: 6682
I20260929 19:07:56.341262 26939 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.78.212:51455
Sep 29, 2026 7:07:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:07:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:07:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 7:07:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 7:07:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 7:07:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 19:07:58.640133 26939 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.224s user 0.008s sys 0.141s
I20260929 19:07:58.640237 26939 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 19:07:58.640432 26939 mini_ranger_kms.cc:401] 127.26.78.212:51455/kms/v1/keys
Sep 29 19:07:58 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790708878, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.78.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 19:07:59 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708878, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.78.212@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
Sep 29 19:07:59 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708878, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.78.212@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
Sep 29 19:08:01 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:08:01.857532 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36199
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:36199
--ranger_config_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:01.938400 6733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:01.938606 6733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:01.938627 6733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:01.939739 6733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:01.939772 6733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:01.939786 6733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:01.939800 6733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:01.939811 6733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:01.941286 6733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.78.212:51455/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:36199
--ranger_config_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:01.941478 6733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:01.941645 6733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:01.947036 6741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:01.946846 6739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:01.946846 6733 server_base.cc:1034] running on GCE node
W20260929 19:08:01.946904 6738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:01.947402 6733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:01.947713 6733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:01.948889 6733 hybrid_clock.cc:648] HybridClock initialized: now 1790708881948881 us; error 37 us; skew 500 ppm
Sep 29 19:08:01 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708881, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:01.951403 6733 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:01.951800 6733 webserver.cc:460] Webserver started at http://127.26.78.254:42587/ using document root <none> and password file <none>
I20260929 19:08:01.951999 6733 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:01.952036 6733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:01.952178 6733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:08:01 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708881, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:02.006031 6733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "66fe0a3c686d4d61bbbc57219048bf3b"
format_stamp: "Formatted at 2026-09-29 19:08:01 on dist-test-slave-xz51"
server_key: "40be67cd7ca6045da8396b3024835bc8"
server_key_iv: "12ea558fb2e0571f3b782d2ed1d1c471"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:02.006359 6733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "66fe0a3c686d4d61bbbc57219048bf3b"
format_stamp: "Formatted at 2026-09-29 19:08:01 on dist-test-slave-xz51"
server_key: "40be67cd7ca6045da8396b3024835bc8"
server_key_iv: "12ea558fb2e0571f3b782d2ed1d1c471"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:02.007754 6733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:08:02.022367 6748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:02.022673 6733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:02.022794 6733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "66fe0a3c686d4d61bbbc57219048bf3b"
format_stamp: "Formatted at 2026-09-29 19:08:01 on dist-test-slave-xz51"
server_key: "40be67cd7ca6045da8396b3024835bc8"
server_key_iv: "12ea558fb2e0571f3b782d2ed1d1c471"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:02.022912 6733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:02.039518 6733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:02.041188 6733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:02.041355 6733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:02.045802 6733 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:36199
I20260929 19:08:02.045867 6800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:36199 every 8 connection(s)
I20260929 19:08:02.046226 6733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:02.046837 6801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:02.049243 6801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Bootstrap starting.
I20260929 19:08:02.049876 6801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:02.050174 6801 log.cc:824] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:02.050860 6801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: No bootstrap required, opened a new log
I20260929 19:08:02.052917 6801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:02.053093 6801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:02.053140 6801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fe0a3c686d4d61bbbc57219048bf3b, State: Initialized, Role: FOLLOWER
I20260929 19:08:02.053308 6801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:02.053409 6801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:02.053452 6801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:02.053503 6801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:02.053695 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 6733
I20260929 19:08:02.053798 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:08:02.054000 26939 external_mini_cluster.cc:1428] Setting key 6a944de7568c2e778213411a0ea971e2
I20260929 19:08:02.054241 6801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:02.054361 6801 leader_election.cc:302] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66fe0a3c686d4d61bbbc57219048bf3b; no voters:
I20260929 19:08:02.054561 6801 leader_election.cc:288] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:08:02.054656 6805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:02.054804 6805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 LEADER]: Becoming Leader. State: Replica: 66fe0a3c686d4d61bbbc57219048bf3b, State: Running, Role: LEADER
I20260929 19:08:02.054811 6801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:02.055032 6805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:02.055519 6807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66fe0a3c686d4d61bbbc57219048bf3b. Latest consensus state: current_term: 1 leader_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } } }
I20260929 19:08:02.055799 6807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:02.055724 6808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } } }
I20260929 19:08:02.056193 6808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:02.056638 6801 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 19:08:02.057396 6815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:02.059917 6815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:08:02 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:02.066756 6815 catalog_manager.cc:1347] Generated new cluster ID: b19c73d3515642828af4e681b1d80a0a
I20260929 19:08:02.066823 6815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:08:02.069844 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:02.054499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.26.78.254:36199)
0929 19:08:02.054843 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:02.054854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:02.054913 (+ 59us) server_negotiation.cc:407] Connection header received
0929 19:08:02.055035 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:02.055043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:02.055102 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:02.055246 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:02.056091 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:02.059513 (+ 3422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:02.061789 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:02.062607 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:02.062745 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:02.065946 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:02.065952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:02.065952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:02.065968 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:02.068121 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:02.068506 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:02.068509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:02.068509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:02.068632 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:02.068837 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:02.068838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:02.068838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:02.069171 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:02.069333 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:02.069441 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 19:08:02.069583 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":55,"threads_started":1}
I20260929 19:08:02.082681 6815 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:08:02.083501 6815 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:02.087850 6815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Generated new TSK 0
I20260929 19:08:02.088045 6815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:08:02.796970 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36199
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:02.882252 6846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:02.882485 6846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:02.882524 6846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:02.883683 6846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:02.883752 6846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:02.883795 6846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:08:02.885298 6846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.78.212:51455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:02.885553 6846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:02.885780 6846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:02.890818 6852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:02.890875 6854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:02.890944 6851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:02.891139 6846 server_base.cc:1034] running on GCE node
I20260929 19:08:02.891390 6846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:02.891645 6846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:02.892808 6846 hybrid_clock.cc:648] HybridClock initialized: now 1790708882892795 us; error 31 us; skew 500 ppm
Sep 29 19:08:02 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708882, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:02.899821 6846 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:08:02.900399 6846 webserver.cc:460] Webserver started at http://127.26.78.193:34725/ using document root <none> and password file <none>
I20260929 19:08:02.900655 6846 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:02.900727 6846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:02.900869 6846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:08:02 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708882, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:02.913079 6846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "644cd802cbca4593837d408c1b16970d"
format_stamp: "Formatted at 2026-09-29 19:08:02 on dist-test-slave-xz51"
server_key: "04c9629bfd3abc5996252bc6558ee46f"
server_key_iv: "6296f8a305a7bf02a6a7617d27a7df28"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:02.913605 6846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "644cd802cbca4593837d408c1b16970d"
format_stamp: "Formatted at 2026-09-29 19:08:02 on dist-test-slave-xz51"
server_key: "04c9629bfd3abc5996252bc6558ee46f"
server_key_iv: "6296f8a305a7bf02a6a7617d27a7df28"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:02.915536 6846 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 19:08:02.929312 6861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:02.929543 6846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:02.929642 6846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "644cd802cbca4593837d408c1b16970d"
format_stamp: "Formatted at 2026-09-29 19:08:02 on dist-test-slave-xz51"
server_key: "04c9629bfd3abc5996252bc6558ee46f"
server_key_iv: "6296f8a305a7bf02a6a7617d27a7df28"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:02.929754 6846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:02.948030 6846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:02.950584 6846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:02.950724 6846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:02.950954 6846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:02.951341 6846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:02.951413 6846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:02.951479 6846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:02.951514 6846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:02.958395 6846 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:41795
I20260929 19:08:02.958590 6976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:41795 every 8 connection(s)
I20260929 19:08:02.958762 6846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:08:02.962229 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 6846
I20260929 19:08:02.962324 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:08:02.962493 26939 external_mini_cluster.cc:1428] Setting key 2ee348b1d7109673bc0f01ec7fa4ce45
Sep 29 19:08:02 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708882, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:02.970659 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:02.959480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36199 (local address 127.26.78.193:53631)
0929 19:08:02.959682 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:02.959730 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:02.960398 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:02.960833 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:02.960838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:02.961103 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:02.961372 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:02.961378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:02.962491 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:02.962492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:02.963779 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:02.963782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:02.964334 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:02.964335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:02.964495 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:02.965134 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:02.965144 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:02.966855 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:02.968927 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:02.968934 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:02.968936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:02.969228 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:02.969463 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:02.969464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:02.969464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:02.969599 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:02.969982 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:02.969989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:02.970157 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 19:08:02.970259 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 19:08:02.970416 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260929 19:08:02.971345 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:02.959549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:53631 (local address 127.26.78.254:36199)
0929 19:08:02.960595 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:02.960597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:02.960605 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:08:02.960633 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:02.960634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:02.960648 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:02.960725 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:02.961492 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:02.962373 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:02.963904 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:02.964214 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:02.964295 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:02.966963 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:02.966965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:02.966965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:02.966972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:02.968835 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:02.969316 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:02.969317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:02.969318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:02.969384 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:02.969676 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:02.969677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:02.969678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:02.969881 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:02.971111 (+ 1230us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:02.971157 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 19:08:02.971251 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1014,"thread_start_us":71,"threads_started":1}
I20260929 19:08:02.971736 6977 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36199
I20260929 19:08:02.971860 6977 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:02.972075 6977 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:02.972626 6765 ts_manager.cc:194] Registered new tserver with Master: 644cd802cbca4593837d408c1b16970d (127.26.78.193:41795)
I20260929 19:08:02.973337 6765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:53631
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:08:02.998858 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36199
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:03.101080 6983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:03.101302 6983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:03.101341 6983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:03.102617 6983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:03.102726 6983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:03.102782 6983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:08:03.104359 6983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.78.212:51455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:03.104609 6983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:03.104883 6983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:03.110435 6993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:03.110808 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:03.110427 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:03.110630 6983 server_base.cc:1034] running on GCE node
I20260929 19:08:03.111140 6983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:03.111408 6983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:03.112577 6983 hybrid_clock.cc:648] HybridClock initialized: now 1790708883112562 us; error 33 us; skew 500 ppm
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:03.115674 6983 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:08:03.116212 6983 webserver.cc:460] Webserver started at http://127.26.78.194:44175/ using document root <none> and password file <none>
I20260929 19:08:03.116437 6983 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:03.116506 6983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:03.116647 6983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:03.135617 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "ef290dc482b2478ab8502a26b8a39d08"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "77aeaceda7788977efd5324ee675e576"
server_key_iv: "f1852982633f81a10ea9b32479345821"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.136297 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "ef290dc482b2478ab8502a26b8a39d08"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "77aeaceda7788977efd5324ee675e576"
server_key_iv: "f1852982633f81a10ea9b32479345821"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.142083 6983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260929 19:08:03.165992 7000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.166317 6983 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260929 19:08:03.166455 6983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "ef290dc482b2478ab8502a26b8a39d08"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "77aeaceda7788977efd5324ee675e576"
server_key_iv: "f1852982633f81a10ea9b32479345821"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.166558 6983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:03.181474 6983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:03.184015 6983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:03.184211 6983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:03.184509 6983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:03.184947 6983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:03.185031 6983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.185097 6983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:03.185146 6983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.193010 6983 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:36569
I20260929 19:08:03.193172 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:36569 every 8 connection(s)
I20260929 19:08:03.193498 6983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:08:03.195072 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 6983
I20260929 19:08:03.195156 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:08:03.195407 26939 external_mini_cluster.cc:1428] Setting key 5d8486c78d52a35dc5ff1864cc5fcf5c
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:03.210117 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:03.194705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:47649 (local address 127.26.78.254:36199)
0929 19:08:03.194758 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:03.194761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:03.198733 (+ 3972us) server_negotiation.cc:407] Connection header received
0929 19:08:03.199469 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:03.199471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:03.199485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:03.199521 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:03.200564 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.201350 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.203855 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.204160 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.204209 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:03.206729 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:03.206730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:03.206730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:03.206740 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:03.208361 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:03.208841 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:03.208842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:03.208842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:03.208900 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:03.209180 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:03.209181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:03.209181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:03.209331 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:03.209392 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:03.209919 (+ 527us) server_negotiation.cc:299] Negotiation successful
0929 19:08:03.210013 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:08:03.210517 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:03.194398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36199 (local address 127.26.78.194:47649)
0929 19:08:03.194600 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:03.198603 (+ 4003us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:03.199329 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:03.199874 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:03.199879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:03.200127 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:03.200444 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:03.200451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.201475 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.201477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:03.203717 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:03.203720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.204318 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.204332 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:03.204478 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:03.205120 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:03.205132 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:03.206625 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:03.208463 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:03.208470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:03.208473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:03.208753 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:03.208966 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:03.208966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:03.208967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:03.209102 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:03.209419 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:03.209426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:03.209587 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 19:08:03.209691 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 19:08:03.210272 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260929 19:08:03.211345 7114 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36199
I20260929 19:08:03.211472 7114 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:03.211674 7114 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:03.212059 6765 ts_manager.cc:194] Registered new tserver with Master: ef290dc482b2478ab8502a26b8a39d08 (127.26.78.194:36569)
I20260929 19:08:03.212543 6765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:47649
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:08:03.239835 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36199
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:03.365600 7120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:03.366070 7120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:03.366178 7120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:03.367996 7120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:03.368232 7120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:03.368351 7120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:08:03.370837 7120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.78.212:51455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:03.371305 7120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:03.371667 7120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:03.378587 7126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:03.379894 7120 server_base.cc:1034] running on GCE node
W20260929 19:08:03.382282 7128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:03.386307 7125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:03.386732 7120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:03.387033 7120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:03.388216 7120 hybrid_clock.cc:648] HybridClock initialized: now 1790708883388186 us; error 44 us; skew 500 ppm
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:03.391819 7120 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:08:03.392383 7120 webserver.cc:460] Webserver started at http://127.26.78.195:43349/ using document root <none> and password file <none>
I20260929 19:08:03.392665 7120 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:03.392738 7120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:03.392877 7120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:03.415125 7120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "70b9a5327de3424fb387d0861f234bf9"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "3c100d9997be65c9f6b590ce1aac256c"
server_key_iv: "5e289cfc314fa01911113d546a0e0dd2"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.416033 7120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "70b9a5327de3424fb387d0861f234bf9"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "3c100d9997be65c9f6b590ce1aac256c"
server_key_iv: "5e289cfc314fa01911113d546a0e0dd2"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.419312 7120 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260929 19:08:03.439505 7135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.439766 7120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260929 19:08:03.439927 7120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "70b9a5327de3424fb387d0861f234bf9"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "3c100d9997be65c9f6b590ce1aac256c"
server_key_iv: "5e289cfc314fa01911113d546a0e0dd2"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.440107 7120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:03.467082 7120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:03.469316 7120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:03.469487 7120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:03.469715 7120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:03.470180 7120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:03.470237 7120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.470278 7120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:03.470319 7120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.477581 7120 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:40845
I20260929 19:08:03.478035 7120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:03.487468 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:03.478819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36199 (local address 127.26.78.195:59989)
0929 19:08:03.478974 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:03.479017 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:03.479524 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:03.479792 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:03.479797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:03.479994 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:03.480189 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:03.480193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.480911 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.480912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:03.481722 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:03.481724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.482317 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.482318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:03.482453 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:03.483009 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:03.483018 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:03.484281 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:03.485984 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:03.485990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:03.485993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:03.486218 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:03.486452 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:03.486453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:03.486453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:03.486569 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:03.486830 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:03.486836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:03.486983 (+ 147us) client_negotiation.cc:769] Sending connection context
0929 19:08:03.487089 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 19:08:03.487236 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260929 19:08:03.488131 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7120
I20260929 19:08:03.488240 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:08:03.488435 26939 external_mini_cluster.cc:1428] Setting key 163a27b3bd944fe3dc9fbae430860f46
I20260929 19:08:03.490078 7248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:40845 every 8 connection(s)
I20260929 19:08:03.506456 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:03.478827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:59989 (local address 127.26.78.254:36199)
0929 19:08:03.478884 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:03.478887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:03.479093 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:08:03.479639 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:03.479641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:03.479653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:03.479709 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:03.480299 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.480815 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.481910 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.482203 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.482307 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:03.484394 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:03.484395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:03.484400 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:03.484407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:03.485906 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:03.486316 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:03.486317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:03.486318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:03.486378 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:03.486636 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:03.486637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:03.486638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:03.486755 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:03.486841 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:03.487122 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 19:08:03.487206 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:08:03.507148 7249 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36199
I20260929 19:08:03.507268 7249 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:03.507499 7249 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:03.507901 6765 ts_manager.cc:194] Registered new tserver with Master: 70b9a5327de3424fb387d0861f234bf9 (127.26.78.195:40845)
I20260929 19:08:03.508478 6765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:59989
I20260929 19:08:03.516825 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:08:03.517479 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 6846
I20260929 19:08:03.523594 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 6983
I20260929 19:08:03.528723 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7120
I20260929 19:08:03.534457 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 6733
I20260929 19:08:03.540849 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36199
--webserver_interface=127.26.78.254
--webserver_port=42587
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:36199
--ranger_config_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:03.624186 7252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:03.624426 7252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:03.624464 7252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:03.625646 7252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:03.625720 7252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:03.625746 7252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:03.625778 7252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:03.625835 7252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:03.627411 7252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:36199
--ranger_config_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=42587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:03.627678 7252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:03.627904 7252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:03.633500 7260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:03.633414 7258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:03.633414 7257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:03.633769 7252 server_base.cc:1034] running on GCE node
I20260929 19:08:03.634024 7252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:03.634326 7252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:03.635496 7252 hybrid_clock.cc:648] HybridClock initialized: now 1790708883635473 us; error 36 us; skew 500 ppm
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:03.638446 7252 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:03.638940 7252 webserver.cc:460] Webserver started at http://127.26.78.254:42587/ using document root <none> and password file <none>
I20260929 19:08:03.639163 7252 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:03.639240 7252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:03.690243 7252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:03.690996 7268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.691221 7252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:03.691335 7252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "66fe0a3c686d4d61bbbc57219048bf3b"
format_stamp: "Formatted at 2026-09-29 19:08:01 on dist-test-slave-xz51"
server_key: "40be67cd7ca6045da8396b3024835bc8"
server_key_iv: "12ea558fb2e0571f3b782d2ed1d1c471"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.691800 7252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:03.708945 7252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:03.711037 7252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:03.711203 7252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:03.715696 7252 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:36199
I20260929 19:08:03.715794 7320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:36199 every 8 connection(s)
I20260929 19:08:03.716126 7252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:03.717280 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7252
I20260929 19:08:03.717396 7321 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:08:03.717597 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:41795
--local_ip_for_outbound_sockets=127.26.78.193
--tserver_master_addrs=127.26.78.254:36199
--webserver_port=34725
--webserver_interface=127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:03.718222 7321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Bootstrap starting.
I20260929 19:08:03.726851 7321 log.cc:824] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:03.728660 7321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:08:03.728979 7321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Bootstrap complete.
I20260929 19:08:03.731405 7321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:03.731700 7321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fe0a3c686d4d61bbbc57219048bf3b, State: Initialized, Role: FOLLOWER
I20260929 19:08:03.731884 7321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:03.731958 7321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:03.732004 7321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:03.732048 7321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:08:03.733171 7321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:03.733297 7321 leader_election.cc:302] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66fe0a3c686d4d61bbbc57219048bf3b; no voters:
I20260929 19:08:03.733501 7321 leader_election.cc:288] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:08:03.733608 7325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:08:03.733772 7325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [term 2 LEADER]: Becoming Leader. State: Replica: 66fe0a3c686d4d61bbbc57219048bf3b, State: Running, Role: LEADER
I20260929 19:08:03.733940 7325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } }
I20260929 19:08:03.733770 7321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:03.734201 7327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } } }
I20260929 19:08:03.734261 7327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:03.734493 7326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66fe0a3c686d4d61bbbc57219048bf3b. Latest consensus state: current_term: 2 leader_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66fe0a3c686d4d61bbbc57219048bf3b" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36199 } } }
I20260929 19:08:03.734616 7326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:03.734786 7329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:03.735376 7329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:03.735625 7329 catalog_manager.cc:1259] Loaded cluster ID: b19c73d3515642828af4e681b1d80a0a
I20260929 19:08:03.735716 7329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:08:03.735744 7321 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 19:08:03.737138 7329 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:03.737370 7329 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Loaded TSK: 0
I20260929 19:08:03.737620 7329 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:08:03.836644 7323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:03.837003 7323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:03.837047 7323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:03.838964 7323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:03.839031 7323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:03.839075 7323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:08:03.841265 7323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:41795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=34725
--tserver_master_addrs=127.26.78.254:36199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:03.841513 7323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:03.841780 7323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:03.848441 7359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:03.848488 7361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:03.848598 7358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:03.849205 7323 server_base.cc:1034] running on GCE node
I20260929 19:08:03.849447 7323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:03.849756 7323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:03.850951 7323 hybrid_clock.cc:648] HybridClock initialized: now 1790708883850916 us; error 46 us; skew 500 ppm
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:03.854667 7323 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:08:03.855283 7323 webserver.cc:460] Webserver started at http://127.26.78.193:34725/ using document root <none> and password file <none>
I20260929 19:08:03.855582 7323 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:03.855671 7323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:03.879288 7323 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260929 19:08:03.880347 7369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.880544 7323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:03.880640 7323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "644cd802cbca4593837d408c1b16970d"
format_stamp: "Formatted at 2026-09-29 19:08:02 on dist-test-slave-xz51"
server_key: "04c9629bfd3abc5996252bc6558ee46f"
server_key_iv: "6296f8a305a7bf02a6a7617d27a7df28"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:03.881037 7323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:03.896103 7323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:03.898579 7323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:03.898763 7323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:03.899044 7323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:03.899475 7323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:03.899593 7323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.899719 7323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:03.899758 7323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:03.907353 7323 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:41795
I20260929 19:08:03.907806 7323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:08:03.908177 7482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:41795 every 8 connection(s)
Sep 29 19:08:03 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:03.917363 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7323
I20260929 19:08:03.917691 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:36569
--local_ip_for_outbound_sockets=127.26.78.194
--tserver_master_addrs=127.26.78.254:36199
--webserver_port=44175
--webserver_interface=127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:03.924129 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:03.908676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36199 (local address 127.26.78.193:33281)
0929 19:08:03.908850 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:03.908902 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:03.909553 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:03.910109 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:03.910114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:03.910330 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:03.910516 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:03.910521 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.911671 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.911672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:03.912472 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:03.912474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.913107 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.913109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:03.913229 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:03.913816 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:03.913874 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:03.915647 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:03.920382 (+ 4735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:03.920393 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:03.920396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:03.920779 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:03.922718 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:03.922719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:03.922719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:03.922869 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:03.923358 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:03.923364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:03.923514 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 19:08:03.923614 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 19:08:03.923880 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260929 19:08:03.925736 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:03.909108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:33281 (local address 127.26.78.254:36199)
0929 19:08:03.909443 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:03.909453 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:03.909488 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:08:03.909810 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:03.909815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:03.909927 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:03.910067 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:03.910639 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.911555 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.912568 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:03.912982 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:03.913116 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:03.915816 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:03.915821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:03.915821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:03.915834 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:03.918663 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:03.922480 (+ 3817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:03.922485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:03.922485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:03.922615 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:03.922952 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:03.922953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:03.922953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:03.923270 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:03.924419 (+ 1149us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:03.924501 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 19:08:03.925455 (+ 954us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":81,"threads_started":1}
I20260929 19:08:03.927932 7483 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36199
I20260929 19:08:03.928033 7483 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:03.928233 7483 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
W20260929 19:08:03.929210 7483 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:36199 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 19:08:03.929288 7483 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:03.929355 7483 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:03.929834 7483 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:03.929916 7483 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
W20260929 19:08:03.930356 7483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:08:04.042327 7487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:04.042582 7487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:04.042629 7487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:04.044528 7487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:04.044607 7487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:04.044658 7487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:08:04.047049 7487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:36569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=44175
--tserver_master_addrs=127.26.78.254:36199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:04.047374 7487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:04.047654 7487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:04.054881 7492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:04.054930 7495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:04.054900 7493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:04.055586 7487 server_base.cc:1034] running on GCE node
I20260929 19:08:04.055845 7487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:04.056135 7487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:04.057343 7487 hybrid_clock.cc:648] HybridClock initialized: now 1790708884057321 us; error 33 us; skew 500 ppm
Sep 29 19:08:04 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:04.060235 7487 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:08:04.060779 7487 webserver.cc:460] Webserver started at http://127.26.78.194:44175/ using document root <none> and password file <none>
I20260929 19:08:04.061048 7487 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:04.061131 7487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:08:04 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:04.082278 7487 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260929 19:08:04.083483 7503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:04.083710 7487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:04.083818 7487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "ef290dc482b2478ab8502a26b8a39d08"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "77aeaceda7788977efd5324ee675e576"
server_key_iv: "f1852982633f81a10ea9b32479345821"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:04.084194 7487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:04.113432 7487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:04.115653 7487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:04.115826 7487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:04.116086 7487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:04.116446 7487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:04.116503 7487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:04.116539 7487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:04.116566 7487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:04.123963 7487 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:36569
I20260929 19:08:04.124060 7616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:36569 every 8 connection(s)
I20260929 19:08:04.124430 7487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:08:04.125419 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7487
I20260929 19:08:04.125702 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.78.212:51455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:40845
--local_ip_for_outbound_sockets=127.26.78.195
--tserver_master_addrs=127.26.78.254:36199
--webserver_port=43349
--webserver_interface=127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:08:04 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:04.135957 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:04.125604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:53347 (local address 127.26.78.254:36199)
0929 19:08:04.125679 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:04.125681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:04.125694 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:08:04.126704 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:04.126706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:04.126723 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:04.126758 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:04.127628 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.128454 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.130003 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.130298 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.130377 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:04.132756 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:04.132758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:04.132758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:04.132766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:04.134417 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:04.134894 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:04.134896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:04.134896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:04.134961 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:04.135215 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:04.135216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:04.135216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:04.135388 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:04.135469 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:04.135756 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 19:08:04.135859 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:08:04.136353 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:04.125346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36199 (local address 127.26.78.194:53347)
0929 19:08:04.125540 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:04.125590 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:04.126559 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:04.126954 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:04.126958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:04.127202 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:04.127502 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:04.127510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.128572 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.128573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:04.129903 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:04.129906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.130464 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.130465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:04.130608 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:04.131219 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:04.131229 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:04.132652 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:04.134514 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:04.134520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:04.134523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:04.134758 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:04.135013 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:04.135014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:04.135015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:04.135135 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:04.135459 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:04.135466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:04.135618 (+ 152us) client_negotiation.cc:769] Sending connection context
0929 19:08:04.135736 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 19:08:04.135895 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260929 19:08:04.136978 7617 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36199
I20260929 19:08:04.137112 7617 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:04.137306 7617 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
W20260929 19:08:04.138006 7617 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:36199 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 19:08:04.138099 7617 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:04.138175 7617 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:04.138478 7617 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:04.138554 7617 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
W20260929 19:08:04.138818 7617 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:08:04.249298 7620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:04.249571 7620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:04.249612 7620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:04.250959 7620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:04.251034 7620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:04.251089 7620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:08:04.252683 7620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.78.212:51455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:40845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=43349
--tserver_master_addrs=127.26.78.254:36199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:04.252947 7620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:04.253180 7620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:04.258275 7627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:04.258289 7626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:04.258800 7620 server_base.cc:1034] running on GCE node
W20260929 19:08:04.261977 7629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:04.262274 7620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:04.262612 7620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:04.263821 7620 hybrid_clock.cc:648] HybridClock initialized: now 1790708884263794 us; error 62 us; skew 500 ppm
Sep 29 19:08:04 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:04.267185 7620 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:08:04.267696 7620 webserver.cc:460] Webserver started at http://127.26.78.195:43349/ using document root <none> and password file <none>
I20260929 19:08:04.268041 7620 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:04.268127 7620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:08:04 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:04.284773 7620 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:04.285753 7637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:04.285984 7620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:04.286073 7620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "70b9a5327de3424fb387d0861f234bf9"
format_stamp: "Formatted at 2026-09-29 19:08:03 on dist-test-slave-xz51"
server_key: "3c100d9997be65c9f6b590ce1aac256c"
server_key_iv: "5e289cfc314fa01911113d546a0e0dd2"
server_key_version: "kuduclusterkey@0"
I20260929 19:08:04.286461 7620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:04.304844 7620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:04.307119 7620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:04.307303 7620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:04.307574 7620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:04.307961 7620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:04.308017 7620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:04.308059 7620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:04.308085 7620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:04.315424 7620 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:40845
I20260929 19:08:04.315507 7750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:40845 every 8 connection(s)
I20260929 19:08:04.315850 7620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:08:04 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:04.323637 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7620
I20260929 19:08:04.327102 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:04.316671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:48085 (local address 127.26.78.254:36199)
0929 19:08:04.316747 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:04.316749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:04.316925 (+ 176us) server_negotiation.cc:407] Connection header received
0929 19:08:04.317462 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:04.317464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:04.317476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:04.317563 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:04.318195 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.318959 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.319963 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.320262 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.320346 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:04.323819 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:04.323821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:04.323821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:04.323830 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:04.325485 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:04.326007 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:04.326008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:04.326008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:04.326076 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:04.326350 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:04.326351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:04.326351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:04.326486 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:04.326561 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:04.326904 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 19:08:04.327009 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:08:04.327530 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:04.316626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36199 (local address 127.26.78.195:48085)
0929 19:08:04.316780 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:04.316822 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:04.317344 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:04.317617 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:04.317620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:04.317795 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:04.318086 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:04.318091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.319062 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.319063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:04.319863 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:04.319864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.320371 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.320372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:04.320549 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:04.322008 (+ 1459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:04.322018 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:04.323704 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:04.325598 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:04.325606 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:04.325609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:04.325921 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:04.326146 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:04.326147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:04.326147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:04.326276 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:04.326566 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:04.326573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:04.326728 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 19:08:04.326866 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 19:08:04.327028 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260929 19:08:04.328028 7751 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36199
I20260929 19:08:04.328148 7751 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:04.328346 7751 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
W20260929 19:08:04.329082 7751 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:36199 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 19:08:04.329211 7751 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:04.329317 7751 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:04.329661 7751 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:04.329746 7751 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
W20260929 19:08:04.330109 7751 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:08:04.335284 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:04.324263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.26.78.254:36199)
0929 19:08:04.327236 (+ 2973us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:04.327238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:04.327245 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:08:04.327264 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:04.327265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:04.327276 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:04.327331 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:04.329099 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.329580 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.330635 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:04.330842 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:04.330934 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:04.332308 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:04.332310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:04.332310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:04.332317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:04.334017 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:04.334462 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:04.334464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:04.334465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:04.334521 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:04.334764 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:04.334765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:04.334765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:04.334885 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:04.335064 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:04.335100 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 19:08:04.335178 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2957,"thread_start_us":69,"threads_started":1}
Sep 29 19:08:04 dist-test-slave-xz51 krb5kdc[6289](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:04.930564 7483 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:04.930737 7483 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:04.931166 7285 ts_manager.cc:194] Registered new tserver with Master: 644cd802cbca4593837d408c1b16970d (127.26.78.193:41795)
I20260929 19:08:04.932191 7285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:33281
I20260929 19:08:05.139036 7617 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:05.139209 7617 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:05.139701 7285 ts_manager.cc:194] Registered new tserver with Master: ef290dc482b2478ab8502a26b8a39d08 (127.26.78.194:36569)
I20260929 19:08:05.140120 7285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:53347
Sep 29 19:08:05 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
I20260929 19:08:05.330293 7751 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:05.330471 7751 heartbeater.cc:511] Master 127.26.78.254:36199 requested a full tablet report, sending...
I20260929 19:08:05.330811 7285 ts_manager.cc:194] Registered new tserver with Master: 70b9a5327de3424fb387d0861f234bf9 (127.26.78.195:40845)
I20260929 19:08:05.331341 7285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:48085
I20260929 19:08:05.337834 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:08:05.350646 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.338798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.26.78.254:36199)
0929 19:08:05.338932 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:05.338935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:05.343040 (+ 4105us) server_negotiation.cc:407] Connection header received
0929 19:08:05.343098 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:05.343099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:05.343114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:05.343196 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:05.343854 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.344667 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.345633 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.345965 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.346032 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.347547 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:05.347549 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:05.347549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:05.347557 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:05.349330 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:05.349741 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:05.349742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:05.349743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:05.349806 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:05.350083 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:05.350083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:05.350084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:05.350221 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:05.350319 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:05.350458 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 19:08:05.350541 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260929 19:08:05.352994 7285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:08:05.355229 7285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:08:05.366963 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.358342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:40845 (local address 127.0.0.1:45200)
0929 19:08:05.362262 (+ 3920us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:05.362269 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:05.362304 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:05.362580 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:05.362581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:05.362583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:05.362635 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.362636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.364209 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.364210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.365353 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.365356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.366683 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.366684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.366749 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.366754 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:05.366801 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:08:05.366860 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3910,"thread_start_us":60,"threads_started":1}
I20260929 19:08:05.366986 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.358455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.26.78.193:41795)
0929 19:08:05.358604 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:05.358609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:05.362253 (+ 3644us) server_negotiation.cc:407] Connection header received
0929 19:08:05.362299 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:05.362302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:05.362321 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:05.362397 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:05.362674 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.363787 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.365477 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.366404 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.366459 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.366528 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:05.366824 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 19:08:05.366889 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260929 19:08:05.367205 7778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.358137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:41795 (local address 127.0.0.1:41696)
0929 19:08:05.362150 (+ 4013us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:05.362158 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:05.362204 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:05.362510 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:05.362513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:05.362516 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:05.362584 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.362586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.363923 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.363924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.365361 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.365365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.366598 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.366599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.366690 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.366705 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 19:08:05.367084 (+ 379us) client_negotiation.cc:240] Negotiation successful
0929 19:08:05.367132 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4001,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260929 19:08:05.367708 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.358580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.26.78.195:40845)
0929 19:08:05.358693 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:05.358698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:05.362361 (+ 3663us) server_negotiation.cc:407] Connection header received
0929 19:08:05.362403 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:05.362405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:05.362422 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:05.362488 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:05.362703 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.364115 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.365451 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.366530 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.366597 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.366650 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:05.367587 (+ 937us) server_negotiation.cc:299] Negotiation successful
0929 19:08:05.367631 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260929 19:08:05.368340 7399 tablet_service.cc:1467] Processing CreateTablet for tablet 9e11e9cd5844495b8d81ff78158b9eb5 (DEFAULT_TABLE table=test-table [id=b6f4d62747f54c9b81dbb0dc1995f63f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:05.368721 7399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e11e9cd5844495b8d81ff78158b9eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:05.370895 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.358130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:36569 (local address 127.0.0.1:34900)
0929 19:08:05.361945 (+ 3815us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:05.361988 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:05.362052 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:05.367361 (+ 5309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:05.367363 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:05.367365 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:05.367407 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.367409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.368418 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.368419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.369580 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.369583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.370687 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.370688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.370746 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.370750 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:08:05.370778 (+ 28us) client_negotiation.cc:240] Negotiation successful
0929 19:08:05.370814 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3782,"thread_start_us":83,"threads_started":1}
I20260929 19:08:05.371199 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.358310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.26.78.194:36569)
0929 19:08:05.367173 (+ 8863us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:05.367178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:05.367190 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:08:05.367227 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:05.367230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:05.367248 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:05.367374 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:05.367466 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.368312 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.369690 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.370568 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.370996 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.371064 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:05.371090 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 19:08:05.371130 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8823,"thread_start_us":61,"threads_started":1}
I20260929 19:08:05.371944 7542 tablet_service.cc:1467] Processing CreateTablet for tablet 9e11e9cd5844495b8d81ff78158b9eb5 (DEFAULT_TABLE table=test-table [id=b6f4d62747f54c9b81dbb0dc1995f63f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:05.372270 7542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e11e9cd5844495b8d81ff78158b9eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:05.373343 7784 tablet_bootstrap.cc:492] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d: Bootstrap starting.
I20260929 19:08:05.374001 7784 tablet_bootstrap.cc:654] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:05.374279 7784 log.cc:824] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:05.374989 7784 tablet_bootstrap.cc:492] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d: No bootstrap required, opened a new log
I20260929 19:08:05.375044 7784 ts_tablet_manager.cc:1397] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:08:05.376415 7685 tablet_service.cc:1467] Processing CreateTablet for tablet 9e11e9cd5844495b8d81ff78158b9eb5 (DEFAULT_TABLE table=test-table [id=b6f4d62747f54c9b81dbb0dc1995f63f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:05.376753 7685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e11e9cd5844495b8d81ff78158b9eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:05.377121 7784 raft_consensus.cc:348] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.377271 7784 raft_consensus.cc:374] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:05.377300 7784 raft_consensus.cc:729] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 644cd802cbca4593837d408c1b16970d, State: Initialized, Role: FOLLOWER
I20260929 19:08:05.377396 7784 consensus_queue.cc:260] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.377622 7784 ts_tablet_manager.cc:1428] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:08:05.377784 7483 heartbeater.cc:503] Master 127.26.78.254:36199 was elected leader, sending a full tablet report...
I20260929 19:08:05.381270 7788 tablet_bootstrap.cc:492] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Bootstrap starting.
I20260929 19:08:05.381294 7786 tablet_bootstrap.cc:492] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Bootstrap starting.
I20260929 19:08:05.381981 7788 tablet_bootstrap.cc:654] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:05.381999 7786 tablet_bootstrap.cc:654] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:05.382313 7786 log.cc:824] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:05.382434 7788 log.cc:824] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:05.383147 7786 tablet_bootstrap.cc:492] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: No bootstrap required, opened a new log
I20260929 19:08:05.383199 7788 tablet_bootstrap.cc:492] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: No bootstrap required, opened a new log
I20260929 19:08:05.383215 7786 ts_tablet_manager.cc:1397] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:08:05.383251 7788 ts_tablet_manager.cc:1397] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:08:05.385654 7788 raft_consensus.cc:348] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.385804 7788 raft_consensus.cc:374] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:05.385875 7788 raft_consensus.cc:729] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b9a5327de3424fb387d0861f234bf9, State: Initialized, Role: FOLLOWER
I20260929 19:08:05.385748 7786 raft_consensus.cc:348] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.386183 7786 raft_consensus.cc:374] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:05.386227 7786 raft_consensus.cc:729] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef290dc482b2478ab8502a26b8a39d08, State: Initialized, Role: FOLLOWER
I20260929 19:08:05.386364 7786 consensus_queue.cc:260] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.386435 7788 consensus_queue.cc:260] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.386704 7786 ts_tablet_manager.cc:1428] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:08:05.386749 7788 ts_tablet_manager.cc:1428] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:08:05.386886 7617 heartbeater.cc:503] Master 127.26.78.254:36199 was elected leader, sending a full tablet report...
I20260929 19:08:05.386909 7751 heartbeater.cc:503] Master 127.26.78.254:36199 was elected leader, sending a full tablet report...
W20260929 19:08:05.410363 7484 tablet.cc:2369] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 19:08:05 dist-test-slave-xz51 krb5kdc[6289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for HTTP/127.26.78.212@KRBTEST.COM
W20260929 19:08:05.567263 7752 tablet.cc:2369] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:08:05.570103 7791 raft_consensus.cc:482] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:05.570235 7791 raft_consensus.cc:504] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.570626 7791 leader_election.cc:288] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 644cd802cbca4593837d408c1b16970d (127.26.78.193:41795), 70b9a5327de3424fb387d0861f234bf9 (127.26.78.195:40845)
I20260929 19:08:05.575083 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.571134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:47991 (local address 127.26.78.193:41795)
0929 19:08:05.571198 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:05.571200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:05.571540 (+ 340us) server_negotiation.cc:407] Connection header received
0929 19:08:05.571653 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:05.571654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:05.571665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:05.571697 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:05.571941 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.572666 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.574130 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.574676 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.574729 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.574738 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:05.574972 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 19:08:05.575012 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:08:05.575366 7796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.570845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:41795 (local address 127.26.78.194:47991)
0929 19:08:05.571469 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:05.571481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:05.571571 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:05.571771 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:05.571772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:05.571774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:05.571860 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.571863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.572782 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.572783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.574016 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.574020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.574823 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.574823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.574891 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.574895 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:08:05.575215 (+ 320us) client_negotiation.cc:240] Negotiation successful
0929 19:08:05.575261 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":110,"threads_started":1}
I20260929 19:08:05.576288 7437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e11e9cd5844495b8d81ff78158b9eb5" candidate_uuid: "ef290dc482b2478ab8502a26b8a39d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "644cd802cbca4593837d408c1b16970d" is_pre_election: true
I20260929 19:08:05.576474 7437 raft_consensus.cc:2454] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef290dc482b2478ab8502a26b8a39d08 in term 0.
I20260929 19:08:05.576817 7504 leader_election.cc:302] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 644cd802cbca4593837d408c1b16970d, ef290dc482b2478ab8502a26b8a39d08; no voters:
I20260929 19:08:05.576987 7791 raft_consensus.cc:2781] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:08:05.577032 7791 raft_consensus.cc:482] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:08:05.577054 7791 raft_consensus.cc:3037] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:05.578166 7791 raft_consensus.cc:504] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.578321 7791 leader_election.cc:288] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [CANDIDATE]: Term 1 election: Requested vote from peers 644cd802cbca4593837d408c1b16970d (127.26.78.193:41795), 70b9a5327de3424fb387d0861f234bf9 (127.26.78.195:40845)
I20260929 19:08:05.578613 7437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e11e9cd5844495b8d81ff78158b9eb5" candidate_uuid: "ef290dc482b2478ab8502a26b8a39d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "644cd802cbca4593837d408c1b16970d"
I20260929 19:08:05.578706 7437 raft_consensus.cc:3037] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:05.579583 7437 raft_consensus.cc:2454] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef290dc482b2478ab8502a26b8a39d08 in term 1.
I20260929 19:08:05.579816 7504 leader_election.cc:302] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 644cd802cbca4593837d408c1b16970d, ef290dc482b2478ab8502a26b8a39d08; no voters:
I20260929 19:08:05.579931 7791 raft_consensus.cc:2781] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:05.580129 7791 raft_consensus.cc:686] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 1 LEADER]: Becoming Leader. State: Replica: ef290dc482b2478ab8502a26b8a39d08, State: Running, Role: LEADER
I20260929 19:08:05.580235 7791 consensus_queue.cc:237] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } }
I20260929 19:08:05.581095 7284 catalog_manager.cc:5495] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef290dc482b2478ab8502a26b8a39d08 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "ef290dc482b2478ab8502a26b8a39d08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef290dc482b2478ab8502a26b8a39d08" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 } health_report { overall_health: UNKNOWN } } }
I20260929 19:08:05.584033 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.571465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:33677 (local address 127.26.78.195:40845)
0929 19:08:05.571516 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:05.571518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:05.574552 (+ 3034us) server_negotiation.cc:407] Connection header received
0929 19:08:05.574654 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:05.574655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:05.574665 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:05.574694 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:05.574926 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.575580 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.583015 (+ 7435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.583591 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.583648 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.583657 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:05.583911 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 19:08:05.583959 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:08:05.584465 7795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.571298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:40845 (local address 127.26.78.194:33677)
0929 19:08:05.574475 (+ 3177us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:05.574488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:05.574581 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:05.574759 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:05.574759 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:05.574762 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:05.574833 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.574835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.576140 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.576141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.582897 (+ 6756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:05.582901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.583725 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.583726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:05.583804 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.583808 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:08:05.584278 (+ 470us) client_negotiation.cc:240] Negotiation successful
0929 19:08:05.584371 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3156,"thread_start_us":90,"threads_started":1}
I20260929 19:08:05.584954 7705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e11e9cd5844495b8d81ff78158b9eb5" candidate_uuid: "ef290dc482b2478ab8502a26b8a39d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b9a5327de3424fb387d0861f234bf9" is_pre_election: true
I20260929 19:08:05.585105 7705 raft_consensus.cc:2454] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef290dc482b2478ab8502a26b8a39d08 in term 0.
I20260929 19:08:05.585233 7704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e11e9cd5844495b8d81ff78158b9eb5" candidate_uuid: "ef290dc482b2478ab8502a26b8a39d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b9a5327de3424fb387d0861f234bf9"
I20260929 19:08:05.585280 7704 raft_consensus.cc:3037] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:05.586154 7704 raft_consensus.cc:2454] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef290dc482b2478ab8502a26b8a39d08 in term 1.
I20260929 19:08:05.605530 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.601938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34908 (local address 127.26.78.194:36569)
0929 19:08:05.601988 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:05.601990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:05.602119 (+ 129us) server_negotiation.cc:407] Connection header received
0929 19:08:05.602207 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:05.602208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:05.602233 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:05.602304 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:08:05.602523 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.603235 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.604366 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:05.604665 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:05.604731 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:05.604947 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:08:05.605090 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:08:05.605171 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:05.605422 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 19:08:05.605444 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:08:05.607683 7437 raft_consensus.cc:1261] T 9e11e9cd5844495b8d81ff78158b9eb5 P 644cd802cbca4593837d408c1b16970d [term 1 FOLLOWER]: Refusing update from remote peer ef290dc482b2478ab8502a26b8a39d08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:05.607683 7704 raft_consensus.cc:1261] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 1 FOLLOWER]: Refusing update from remote peer ef290dc482b2478ab8502a26b8a39d08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:05.608026 7797 consensus_queue.cc:1035] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70b9a5327de3424fb387d0861f234bf9" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:05.608134 7791 consensus_queue.cc:1035] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "644cd802cbca4593837d408c1b16970d" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:05.613245 7800 mvcc.cc:205] Tried to move back new op lower bound from 7334743595445616640 to 7334743595337199616. Current Snapshot: MvccSnapshot[applied={T|T < 7334743595445616640}]
I20260929 19:08:05.613628 7801 mvcc.cc:205] Tried to move back new op lower bound from 7334743595445616640 to 7334743595337199616. Current Snapshot: MvccSnapshot[applied={T|T < 7334743595445616640}]
I20260929 19:08:05.614897 7802 mvcc.cc:205] Tried to move back new op lower bound from 7334743595445616640 to 7334743595337199616. Current Snapshot: MvccSnapshot[applied={T|T < 7334743595445616640}]
I20260929 19:08:05.620123 7284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:05.620231 7284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:05.621083 7284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66fe0a3c686d4d61bbbc57219048bf3b: Sending DeleteTablet for 3 replicas of tablet 9e11e9cd5844495b8d81ff78158b9eb5
I20260929 19:08:05.621444 7399 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e11e9cd5844495b8d81ff78158b9eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:05 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41696
I20260929 19:08:05.621894 7542 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e11e9cd5844495b8d81ff78158b9eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:05 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:34900
I20260929 19:08:05.622423 7685 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e11e9cd5844495b8d81ff78158b9eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:05 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:45200
I20260929 19:08:05.622902 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7323
I20260929 19:08:05.624871 7810 tablet_replica.cc:330] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: stopping tablet replica
I20260929 19:08:05.625015 7810 raft_consensus.cc:2229] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:08:05.625213 7810 raft_consensus.cc:2258] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:05.625433 7811 tablet_replica.cc:330] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: stopping tablet replica
I20260929 19:08:05.625921 7811 raft_consensus.cc:2229] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:05.626098 7810 ts_tablet_manager.cc:1905] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:05.626263 7811 raft_consensus.cc:2258] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:05.628427 7811 ts_tablet_manager.cc:1905] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:05.630095 7811 ts_tablet_manager.cc:1918] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:05.634140 7810 ts_tablet_manager.cc:1918] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:05.634279 7810 log.cc:1196] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/9e11e9cd5844495b8d81ff78158b9eb5
I20260929 19:08:05.634665 7810 ts_tablet_manager.cc:1939] T 9e11e9cd5844495b8d81ff78158b9eb5 P ef290dc482b2478ab8502a26b8a39d08: Deleting consensus metadata
I20260929 19:08:05.634810 7811 log.cc:1196] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/9e11e9cd5844495b8d81ff78158b9eb5
I20260929 19:08:05.635160 7811 ts_tablet_manager.cc:1939] T 9e11e9cd5844495b8d81ff78158b9eb5 P 70b9a5327de3424fb387d0861f234bf9: Deleting consensus metadata
I20260929 19:08:05.635689 7272 catalog_manager.cc:4841] TS ef290dc482b2478ab8502a26b8a39d08 (127.26.78.194:36569): tablet 9e11e9cd5844495b8d81ff78158b9eb5 (table test-table [id=b6f4d62747f54c9b81dbb0dc1995f63f]) successfully deleted
I20260929 19:08:05.636207 7272 catalog_manager.cc:4841] TS 70b9a5327de3424fb387d0861f234bf9 (127.26.78.195:40845): tablet 9e11e9cd5844495b8d81ff78158b9eb5 (table test-table [id=b6f4d62747f54c9b81dbb0dc1995f63f]) successfully deleted
W20260929 19:08:05.636500 7269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:41795 (error 108)
I20260929 19:08:05.637426 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:05.637301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:41795 (local address 127.0.0.1:41706)
0929 19:08:05.637348 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:05.637386 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 19:08:05.637455 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7487
W20260929 19:08:05.639946 7269 catalog_manager.cc:4584] TS 644cd802cbca4593837d408c1b16970d (127.26.78.193:41795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e11e9cd5844495b8d81ff78158b9eb5: Network error: Client connection negotiation failed: client connection to 127.26.78.193:41795: connect: Connection refused (error 111)
I20260929 19:08:05.643538 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7620
I20260929 19:08:05.649533 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7252
I20260929 19:08:05.658286 26939 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 19:08:05.992934 26939 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 19:08:05.993028 26939 mini_ranger.cc:66] Stopping Ranger...
I20260929 19:08:06.337967 26939 mini_ranger.cc:68] Stopped Ranger
2026-09-29 19:08:06.338 UTC [6319] LOG: received smart shutdown request
2026-09-29 19:08:06.342 UTC [6319] LOG: background worker "logical replication launcher" (PID 6328) exited with exit code 1
2026-09-29 19:08:06.343 UTC [6323] LOG: shutting down
2026-09-29 19:08:06.526 UTC [6319] LOG: database system is shut down
2026-09-29T19:08:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48297 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:08:06 dist-test-slave-xz51 krb5kdc[7821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:08:06 dist-test-slave-xz51 krb5kdc[7821](info): set up 2 sockets
Sep 29 19:08:06 dist-test-slave-xz51 krb5kdc[7821](info): commencing operation
krb5kdc: starting...
W20260929 19:08:07.681784 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:08:07 dist-test-slave-xz51 krb5kdc[7821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:08:07.800061 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33161
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:33161,127.26.78.253:44617,127.26.78.252:43529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:07.880199 7834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:07.880411 7834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:07.880434 7834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:07.881613 7834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:07.881657 7834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:07.881672 7834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:07.881685 7834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:07.881697 7834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:07.883348 7834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:33161,127.26.78.253:44617,127.26.78.252:43529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:07.883615 7834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:07.883872 7834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:07.888830 7839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:07.888904 7834 server_base.cc:1034] running on GCE node
W20260929 19:08:07.888878 7840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:07.888890 7842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:07.889478 7834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:07.889766 7834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:07.890954 7834 hybrid_clock.cc:648] HybridClock initialized: now 1790708887890923 us; error 54 us; skew 500 ppm
Sep 29 19:08:07 dist-test-slave-xz51 krb5kdc[7821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:07.893306 7834 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:07.893777 7834 webserver.cc:460] Webserver started at http://127.26.78.254:45423/ using document root <none> and password file <none>
I20260929 19:08:07.894050 7834 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:07.894121 7834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:07.894271 7834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:07.895306 7834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "2b88f21361924a7fa119d04c96f67a88"
format_stamp: "Formatted at 2026-09-29 19:08:07 on dist-test-slave-xz51"
server_key: "f56e8bf2ab383d6ad414582a72492714"
server_key_iv: "fc5481e63f832a07c2d2fb88a0ea0eb1"
server_key_version: "encryptionkey@0"
I20260929 19:08:07.895675 7834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "2b88f21361924a7fa119d04c96f67a88"
format_stamp: "Formatted at 2026-09-29 19:08:07 on dist-test-slave-xz51"
server_key: "f56e8bf2ab383d6ad414582a72492714"
server_key_iv: "fc5481e63f832a07c2d2fb88a0ea0eb1"
server_key_version: "encryptionkey@0"
I20260929 19:08:07.896955 7834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:08:07.897704 7849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:07.897922 7834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:07.898020 7834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "2b88f21361924a7fa119d04c96f67a88"
format_stamp: "Formatted at 2026-09-29 19:08:07 on dist-test-slave-xz51"
server_key: "f56e8bf2ab383d6ad414582a72492714"
server_key_iv: "fc5481e63f832a07c2d2fb88a0ea0eb1"
server_key_version: "encryptionkey@0"
I20260929 19:08:07.898106 7834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:07.915172 7834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:07.917151 7834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:07.917317 7834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:07.921680 7834 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:33161
I20260929 19:08:07.921904 7901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:33161 every 8 connection(s)
I20260929 19:08:07.922164 7834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:07.923045 7902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:07.924677 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7834
I20260929 19:08:07.924788 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:08:07.924988 26939 external_mini_cluster.cc:1428] Setting key df44a1d881121740fe3e720058630d3e
I20260929 19:08:07.924717 7902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:08:07 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.78.253@KRBTEST.COM; defaulting to no policy
I20260929 19:08:07.935413 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:07.925524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33161 (local address 127.0.0.1:45708)
0929 19:08:07.925711 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:07.925755 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:07.926516 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:07.926836 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:07.926838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:07.927089 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:07.927257 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:07.927261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:07.928306 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:07.928307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:07.929006 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:07.929008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:07.929744 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:07.929745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:07.929808 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:07.930378 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:07.930387 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:07.932006 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:07.934079 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:07.934085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:07.934087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:07.934287 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:07.934512 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:07.934512 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:07.934512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:07.934592 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:07.934925 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:07.934929 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:07.934984 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 19:08:07.935099 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 19:08:07.935293 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":64,"thread_start_us":87,"threads_started":1}
I20260929 19:08:07.935420 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:07.925566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45708 (local address 127.26.78.254:33161)
0929 19:08:07.925729 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:07.925739 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:07.925888 (+ 149us) server_negotiation.cc:407] Connection header received
0929 19:08:07.926698 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:07.926702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:07.926725 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:07.926836 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:07.927368 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:07.928216 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:07.929094 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:07.929638 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:07.929768 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:07.932098 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:07.932106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:07.932106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:07.932114 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:07.933999 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:07.934367 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:07.934370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:07.934370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:07.934447 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:07.934649 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:07.934650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:07.934650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:07.934847 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:07.934927 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:07.935130 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 19:08:07.935227 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":63,"threads_started":1}
I20260929 19:08:07.936350 7902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.253@KRBTEST.COM" created.
I20260929 19:08:07.936728 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:07.936608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:60944)
0929 19:08:07.936656 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:07.936705 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 19:08:07.936899 7851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
I20260929 19:08:07.937340 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:07.937228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:60946)
0929 19:08:07.937277 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:07.937309 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 19:08:07.937450 7902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:44617: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab.
Entry for principal kudu/127.26.78.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab.
I20260929 19:08:07.952035 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:44617
--webserver_interface=127.26.78.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab
--principal=kudu/127.26.78.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:33161,127.26.78.253:44617,127.26.78.252:43529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:07.986701 7902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } attempt: 1
I20260929 19:08:07.987280 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:07.987108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:60962)
0929 19:08:07.987175 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:07.987227 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 19:08:07.988173 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:07.988018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:60978)
0929 19:08:07.988083 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:07.988138 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 19:08:07.988336 7902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:44617: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
W20260929 19:08:08.048255 7909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:08.048475 7909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:08.048497 7909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:08.049772 7909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:08.049854 7909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:08.049887 7909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:08.049932 7909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:08.049954 7909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:08.051484 7909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:33161,127.26.78.253:44617,127.26.78.252:43529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab
--principal=kudu/127.26.78.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:44617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.78.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:08.051731 7909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:08.051956 7909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:08:08.056531 7902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } attempt: 2
W20260929 19:08:08.056820 7914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:08.056967 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.056831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:60994)
0929 19:08:08.056898 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.056927 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 19:08:08.056981 7915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:08.057031 7909 server_base.cc:1034] running on GCE node
W20260929 19:08:08.057148 7917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:08.057281 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.057201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:60998)
0929 19:08:08.057241 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.057261 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 19:08:08.057440 7902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:44617: Network error: Client connection negotiation failed: client connection to 127.26.78.253:44617: connect: Connection refused (error 111)
I20260929 19:08:08.057438 7909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:08.057713 7909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:08.058880 7909 hybrid_clock.cc:648] HybridClock initialized: now 1790708888058867 us; error 45 us; skew 500 ppm
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:08.061303 7909 init.cc:377] Logged in from keytab as kudu/127.26.78.253@KRBTEST.COM (short username kudu)
I20260929 19:08:08.061776 7909 webserver.cc:460] Webserver started at http://127.26.78.253:33975/ using document root <none> and password file <none>
I20260929 19:08:08.062069 7909 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:08.062139 7909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:08.062295 7909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:08.063365 7909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data/instance:
uuid: "03eb3313432d4d388193bdb8cada9563"
format_stamp: "Formatted at 2026-09-29 19:08:08 on dist-test-slave-xz51"
server_key: "dd9f8a09f73878036dbe753f089d5ea9"
server_key_iv: "13d111d614e5072d6cd4f1a009445b1d"
server_key_version: "encryptionkey@0"
I20260929 19:08:08.063727 7909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/wal/instance:
uuid: "03eb3313432d4d388193bdb8cada9563"
format_stamp: "Formatted at 2026-09-29 19:08:08 on dist-test-slave-xz51"
server_key: "dd9f8a09f73878036dbe753f089d5ea9"
server_key_iv: "13d111d614e5072d6cd4f1a009445b1d"
server_key_version: "encryptionkey@0"
I20260929 19:08:08.065021 7909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:08:08.065701 7924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:08.065927 7909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:08:08.066026 7909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/wal
uuid: "03eb3313432d4d388193bdb8cada9563"
format_stamp: "Formatted at 2026-09-29 19:08:08 on dist-test-slave-xz51"
server_key: "dd9f8a09f73878036dbe753f089d5ea9"
server_key_iv: "13d111d614e5072d6cd4f1a009445b1d"
server_key_version: "encryptionkey@0"
I20260929 19:08:08.066109 7909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:08.083127 7909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:08.085155 7909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:08.085327 7909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:08.089383 7909 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.253:44617
I20260929 19:08:08.089422 7976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.253:44617 every 8 connection(s)
I20260929 19:08:08.089802 7909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
I20260929 19:08:08.090739 7977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:08.092253 7977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:08:08.097201 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7909
I20260929 19:08:08.097312 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-1/wal/instance
I20260929 19:08:08.097527 26939 external_mini_cluster.cc:1428] Setting key f7b5a023dd12522947945f1522b77483
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:08.101526 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.092942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.26.78.254:33161)
0929 19:08:08.093021 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.093023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.093217 (+ 194us) server_negotiation.cc:407] Connection header received
0929 19:08:08.093666 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.093667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.093677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.093736 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.094434 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.094937 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.095890 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.096085 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.096139 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.098523 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.098525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.098525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.098532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.099983 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.100463 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.100464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.100464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.100518 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.100814 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.100815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.100815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.100927 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.100982 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.101312 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.101417 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 19:08:08.101761 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.092907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33161 (local address 127.0.0.1:45716)
0929 19:08:08.093084 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.093125 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.093585 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.093802 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.093805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.094134 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.094355 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.094360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.095016 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.095017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.095807 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.095809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.096155 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.096156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.096252 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.096702 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.096709 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.098429 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.100062 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.100070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.100072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.100382 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.100607 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.100608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.100609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.100752 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.100995 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.101001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.101171 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.101290 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.101472 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260929 19:08:08.102401 7977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.78.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.252@KRBTEST.COM" created.
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:08:08.112855 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.102670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:32780)
0929 19:08:08.102719 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.102735 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.102772 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.103020 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.103021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.103189 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.103253 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.103255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.104100 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.104102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.104941 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.104944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.105389 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.105390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.105480 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.105886 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.105893 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.109773 (+ 3880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.111683 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.111685 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.111685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.111899 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.112120 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.112120 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.112120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.112169 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.112599 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.112600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.112642 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.112710 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.112786 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 19:08:08.113026 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.102732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.26.78.253:44617)
0929 19:08:08.102883 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.102887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.102901 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:08:08.102920 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.102921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.102932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.102993 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.103329 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.104010 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.105023 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.105314 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.105369 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.109924 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.109934 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.109935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.109943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.111598 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.111984 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.111988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.111988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.112052 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.112230 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.112231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.112232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.112439 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.112525 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.112772 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.112860 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260929 19:08:08.113627 7977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:08:08.114056 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.113917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48860)
0929 19:08:08.113973 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.114027 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 19:08:08.114212 7926 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
I20260929 19:08:08.114681 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.114567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48868)
0929 19:08:08.114617 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.114650 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 19:08:08.114831 7977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:43529: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab.
Entry for principal kudu/127.26.78.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab.
I20260929 19:08:08.124836 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:43529
--webserver_interface=127.26.78.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab
--principal=kudu/127.26.78.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:33161,127.26.78.253:44617,127.26.78.252:43529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:08.148602 7902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } attempt: 3
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:08:08.157223 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.149008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:32796)
0929 19:08:08.149081 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.149092 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.149165 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.149411 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.149413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.149601 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.149679 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.149681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.150401 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.150402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.151205 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.151207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.151575 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.151576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.151655 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.152156 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.152162 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.153870 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.155912 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.155913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.155914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.156175 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.156425 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.156426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.156427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.156522 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.156799 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.156800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.156879 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.156962 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.157083 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 19:08:08.157245 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.149090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.26.78.253:44617)
0929 19:08:08.149157 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.149160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.149171 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:08:08.149234 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.149235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.149305 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.149377 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.149777 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.150324 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.151289 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.151503 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.151555 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.153984 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.153986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.153986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.153995 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.155810 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.156277 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.156278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.156278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.156363 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.156587 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.156588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.156588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.156743 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.156807 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.157003 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.157112 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:08:08.163268 7902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:08:08.163750 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.163589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48876)
0929 19:08:08.163660 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.163706 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 19:08:08.164043 7977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } attempt: 1
I20260929 19:08:08.164203 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.164094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48890)
0929 19:08:08.164143 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.164170 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 19:08:08.164294 7902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:43529: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
I20260929 19:08:08.164379 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.164281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48898)
0929 19:08:08.164323 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.164349 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:08:08.164752 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.164621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48914)
0929 19:08:08.164678 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.164716 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 19:08:08.164845 7977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:43529: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
I20260929 19:08:08.195452 7902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } attempt: 1
I20260929 19:08:08.195992 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.195820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48928)
0929 19:08:08.195897 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.195946 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 19:08:08.196453 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.196342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:48942)
0929 19:08:08.196390 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.196421 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 19:08:08.196552 7902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:43529: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
W20260929 19:08:08.213687 7984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:08.213987 7984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:08.214031 7984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:08.215281 7984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:08.215376 7984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:08.215433 7984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:08.215466 7984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:08.215493 7984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:08.217005 7984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:33161,127.26.78.253:44617,127.26.78.252:43529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab
--principal=kudu/127.26.78.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:43529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.78.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:08.217275 7984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:08.217504 7984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:08.222395 7992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:08.222440 7990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:08.222368 7989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:08.222582 7984 server_base.cc:1034] running on GCE node
I20260929 19:08:08.222843 7984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:08.223129 7984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:08.224279 7984 hybrid_clock.cc:648] HybridClock initialized: now 1790708888224263 us; error 39 us; skew 500 ppm
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:08.226765 7984 init.cc:377] Logged in from keytab as kudu/127.26.78.252@KRBTEST.COM (short username kudu)
I20260929 19:08:08.227236 7984 webserver.cc:460] Webserver started at http://127.26.78.252:44301/ using document root <none> and password file <none>
I20260929 19:08:08.227468 7984 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:08.227535 7984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:08.227689 7984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:08.228760 7984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data/instance:
uuid: "615028f6eaf44d9981fd10aa6616a579"
format_stamp: "Formatted at 2026-09-29 19:08:08 on dist-test-slave-xz51"
server_key: "271d8d76794c661e368236a8797a3ab5"
server_key_iv: "9011cb26f4958105a4f39c203b82af91"
server_key_version: "encryptionkey@0"
I20260929 19:08:08.229127 7984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/wal/instance:
uuid: "615028f6eaf44d9981fd10aa6616a579"
format_stamp: "Formatted at 2026-09-29 19:08:08 on dist-test-slave-xz51"
server_key: "271d8d76794c661e368236a8797a3ab5"
server_key_iv: "9011cb26f4958105a4f39c203b82af91"
server_key_version: "encryptionkey@0"
I20260929 19:08:08.230576 7984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:08:08.231271 7999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:08.231472 7984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:08.231578 7984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/wal
uuid: "615028f6eaf44d9981fd10aa6616a579"
format_stamp: "Formatted at 2026-09-29 19:08:08 on dist-test-slave-xz51"
server_key: "271d8d76794c661e368236a8797a3ab5"
server_key_iv: "9011cb26f4958105a4f39c203b82af91"
server_key_version: "encryptionkey@0"
I20260929 19:08:08.231652 7984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:08.232990 7977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } attempt: 2
I20260929 19:08:08.233385 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.233260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:33336)
0929 19:08:08.233320 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.233353 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 19:08:08.233727 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.233646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:33340)
0929 19:08:08.233688 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.233711 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 19:08:08.233855 7977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:43529: Network error: Client connection negotiation failed: client connection to 127.26.78.252:43529: connect: Connection refused (error 111)
I20260929 19:08:08.242347 7984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:08.244151 7984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:08.244315 7984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:08.248457 7984 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.252:43529
I20260929 19:08:08.248503 8051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.252:43529 every 8 connection(s)
I20260929 19:08:08.248871 7984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
I20260929 19:08:08.249802 8052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:08.251353 8052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:08:08.255249 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 7984
I20260929 19:08:08.255391 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790708710206825-26939-0/minicluster-data/master-2/wal/instance
I20260929 19:08:08.255587 26939 external_mini_cluster.cc:1428] Setting key 0d37a75c53664c341ca81c825350109f
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:08:08.261101 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.252018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.26.78.254:33161)
0929 19:08:08.252061 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.252064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.252264 (+ 200us) server_negotiation.cc:407] Connection header received
0929 19:08:08.252692 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.252693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.252702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.252761 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.253384 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.253955 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.254900 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.255156 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.255233 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.257544 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.257545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.257546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.257553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.259620 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.260045 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.260046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.260046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.260109 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.260392 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.260393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.260394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.260542 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.260612 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.260909 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.261006 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:08:08.261261 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.251966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:33161 (local address 127.0.0.1:50790)
0929 19:08:08.252130 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.252172 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.252611 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.252839 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.252842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.253103 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.253292 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.253296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.254039 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.254040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.254814 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.254816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.255240 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.255240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.255376 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.255958 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.255968 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.257441 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.259722 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.259729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.259731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.259962 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.260206 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.260207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.260207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.260321 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.260611 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.260615 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.260730 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.260843 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.261024 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260929 19:08:08.261796 8052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:08:08.264417 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.255974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33356 (local address 127.26.78.252:43529)
0929 19:08:08.256129 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.256134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.256196 (+ 62us) server_negotiation.cc:407] Connection header received
0929 19:08:08.256254 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.256257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.256278 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.256350 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.256945 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.257703 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.258489 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.258797 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.258893 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.261492 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.261498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.261498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.261505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.263220 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.263571 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.263574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.263574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.263639 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.263853 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.263854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.263855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.264038 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.264185 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.264237 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.264315 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:08:08.269522 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.262237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.26.78.253:44617)
0929 19:08:08.262308 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.262310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.262319 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:08:08.262344 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.262345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.262355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.262404 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.262662 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.263141 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.264033 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.264320 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.264387 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.266769 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.266771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.266771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.266779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.268339 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.268689 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.268690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.268691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.268744 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.268962 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.268963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.268963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.269114 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.269177 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.269363 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.269436 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:08:08.269520 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.262154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:44617 (local address 127.0.0.1:38590)
0929 19:08:08.262207 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.262224 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.262275 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.262404 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.262405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.262534 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.262582 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.262583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.263223 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.263224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.263956 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.263958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.264536 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.264537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.264620 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.265017 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.265024 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.266659 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.268437 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.268438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.268438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.268603 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.268827 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.268828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.268828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.268897 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.269170 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.269171 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.269242 (+ 71us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.269327 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.269423 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 19:08:08.269857 8052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:08.271692 7902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } attempt: 2
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:08:08.274514 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.266570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.26.78.254:33161)
0929 19:08:08.266607 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.266608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.266616 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:08:08.266636 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.266637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.266645 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.266693 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.267287 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.267718 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.268489 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.268692 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.268774 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.271395 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.271396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.271397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.271403 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.273219 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.273589 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.273590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.273590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.273647 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.273918 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.273919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.273919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.274035 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.274232 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.274269 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.274415 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20}
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:08:08.278085 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.270131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:33370)
0929 19:08:08.270171 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.270184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.270251 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.270403 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.270404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.270508 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.270537 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.270538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.271310 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.271311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.271945 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.271948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.272759 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.272759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.272810 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.273199 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.273204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.274599 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.276956 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.276957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.276957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.277180 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.277393 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.277394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.277394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.277460 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.277757 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.277758 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.277796 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.277878 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.277974 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 19:08:08.278126 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.270161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.26.78.252:43529)
0929 19:08:08.270193 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.270195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.270234 (+ 39us) server_negotiation.cc:407] Connection header received
0929 19:08:08.270322 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.270323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.270333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.270390 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.270608 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.271232 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.272009 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.272634 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.272736 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.274670 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.274671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.274671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.274676 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.276867 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.277253 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.277254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.277254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.277310 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.277534 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.277535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.277535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.277657 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.277760 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.277915 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.278020 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":16}
I20260929 19:08:08.280097 8052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: Bootstrap starting.
I20260929 19:08:08.280309 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.271918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:33386)
0929 19:08:08.272164 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.272174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.272223 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.272371 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.272371 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.272547 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.272589 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.272591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.273485 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.273486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.274340 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.274343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.274918 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.274919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.274978 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.275328 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.275332 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.276600 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.279141 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.279142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.279142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.279369 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.279598 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.279598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.279599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.279669 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.279975 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.279976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.280028 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.280102 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.280203 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":3}
I20260929 19:08:08.280336 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.271978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.26.78.252:43529)
0929 19:08:08.272178 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.272180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.272208 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:08:08.272267 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.272268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.272278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.272362 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.272668 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.273386 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.274513 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.274845 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.274919 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.276690 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.276692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.276692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.276699 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.279045 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.279454 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.279456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.279456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.279515 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.279747 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.279748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.279749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.279894 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.279965 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.280132 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.280205 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":145,"threads_started":1}
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:08:08.280891 8052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:08.281224 8052 log.cc:824] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:08.281949 7902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88: Bootstrap starting.
I20260929 19:08:08.282229 8052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: No bootstrap required, opened a new log
I20260929 19:08:08.282693 7902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:08.282989 7902 log.cc:824] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:08.283766 7902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88: No bootstrap required, opened a new log
I20260929 19:08:08.284076 8052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.284245 8052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:08.284283 8052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615028f6eaf44d9981fd10aa6616a579, State: Initialized, Role: FOLLOWER
I20260929 19:08:08.284380 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.275105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.26.78.253:44617)
0929 19:08:08.275151 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.275153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.275192 (+ 39us) server_negotiation.cc:407] Connection header received
0929 19:08:08.275258 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.275259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.275271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.275329 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.276274 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.276904 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.278161 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.278391 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.278457 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.281152 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.281153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.281154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.281161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.282951 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.283384 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.283385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.283385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.283444 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.283660 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.283661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.283662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.283804 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.283984 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.284198 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.284285 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:08:08.284404 8052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.284662 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.284722 8052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:08.284722 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:08:08.286257 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:08:08.286567 7902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.286696 7902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:08.286741 7902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b88f21361924a7fa119d04c96f67a88, State: Initialized, Role: FOLLOWER
I20260929 19:08:08.286868 7902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.287165 7902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:08.287158 8073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.287256 8073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:08:08.286824 8071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:08:08.288465 8071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:08:08.288975 7984 master_runner.cc:186] Error getting master registration for 127.26.78.254:33161: OK, instance_id { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" instance_seqno: 1790708887920310 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:08:08.291200 7984 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 19:08:08.295853 8091 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:08:08.295931 8091 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:08:08.297104 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.288073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.26.78.253:44617)
0929 19:08:08.288157 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.288159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.288166 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:08:08.288184 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.288185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.288194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.288248 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.289078 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.289809 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.290800 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.291083 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.291138 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.293328 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.293329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.293330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.293337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.295329 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.296113 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.296114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.296114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.296170 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.296384 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.296385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.296386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.296511 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.296616 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.296935 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.297012 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 19:08:08.297750 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.287766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.26.78.254:33161)
0929 19:08:08.287808 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.287810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.288958 (+ 1148us) server_negotiation.cc:407] Connection header received
0929 19:08:08.288979 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.288980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.288989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.289036 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.289726 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.290452 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.291517 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.291816 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.291870 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.293350 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.293351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.293351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.293357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.295239 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.296822 (+ 1583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.296823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.296823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.296884 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.297122 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.297122 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.297123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.297266 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.297329 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.297576 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.297661 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:08:08.296361 7834 master_runner.cc:186] Error getting master registration for 127.26.78.253:44617: OK, instance_id { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" instance_seqno: 1790708888088244 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:08:08.299350 7834 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:08:08.299690 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.288114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.26.78.252:43529)
0929 19:08:08.288263 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.288265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.288537 (+ 272us) server_negotiation.cc:407] Connection header received
0929 19:08:08.288640 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.288641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.288652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.288705 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.289492 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.290247 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.291479 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.291789 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.291832 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.293576 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.293577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.293577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.293584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.296491 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.298795 (+ 2304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.298796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.298796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.298853 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.299052 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.299053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.299053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.299180 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.299271 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.299507 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.299591 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260929 19:08:08.325031 7977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } attempt: 3
Sep 29 19:08:08 dist-test-slave-xz51 krb5kdc[7821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:08:08.333989 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.325392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:43529 (local address 127.0.0.1:33394)
0929 19:08:08.325490 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:08.325502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:08.325543 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:08.325880 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:08.325882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:08.326076 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:08.326131 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.326133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.327136 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.327137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.327977 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:08.327979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.328482 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.328483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:08.328545 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.329107 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:08.329113 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:08.330732 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:08.332747 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.332749 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.332749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.332981 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.333214 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:08.333215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:08.333215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:08.333293 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:08.333619 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:08.333620 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:08.333675 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 19:08:08.333750 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:08:08.333879 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 19:08:08.334002 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:08.325640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.26.78.252:43529)
0929 19:08:08.325698 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:08.325700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:08.325708 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:08:08.325727 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:08.325728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:08.325738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:08.325798 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:08.326288 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.327038 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.328071 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:08.328392 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:08.328468 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:08.330823 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:08.330825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:08.330825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:08.330833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:08.332556 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.333066 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.333067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.333068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.333134 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:08.333397 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:08.333398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:08.333398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:08.333544 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:08.333613 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:08.333794 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 19:08:08.333902 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:08:08.335906 7977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: Bootstrap starting.
I20260929 19:08:08.336644 7977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:08.336957 7977 log.cc:824] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:08.337911 7977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: No bootstrap required, opened a new log
I20260929 19:08:08.339918 7977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.340061 7977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:08.340095 7977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03eb3313432d4d388193bdb8cada9563, State: Initialized, Role: FOLLOWER
I20260929 19:08:08.340237 7977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.340586 7977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:08.340540 8094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.340732 8094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:08:08.342468 8105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:08:08.342582 8105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:08:08.345543 7909 master_runner.cc:263] No leader master found from master 03eb3313432d4d388193bdb8cada9563
I20260929 19:08:08.345600 7909 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:08:08.356909 8073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:08.357041 8073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.357271 8073 leader_election.cc:288] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03eb3313432d4d388193bdb8cada9563 (127.26.78.253:44617), 615028f6eaf44d9981fd10aa6616a579 (127.26.78.252:43529)
I20260929 19:08:08.357924 8026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b88f21361924a7fa119d04c96f67a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615028f6eaf44d9981fd10aa6616a579" is_pre_election: true
I20260929 19:08:08.358076 8026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b88f21361924a7fa119d04c96f67a88 in term 0.
I20260929 19:08:08.358240 7951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b88f21361924a7fa119d04c96f67a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03eb3313432d4d388193bdb8cada9563" is_pre_election: true
I20260929 19:08:08.358393 7951 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b88f21361924a7fa119d04c96f67a88 in term 0.
I20260929 19:08:08.358609 7851 leader_election.cc:302] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03eb3313432d4d388193bdb8cada9563, 2b88f21361924a7fa119d04c96f67a88; no voters:
I20260929 19:08:08.358862 8073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:08:08.358923 8073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:08:08.358951 8073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:08.359795 8073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.359913 8073 leader_election.cc:288] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [CANDIDATE]: Term 1 election: Requested vote from peers 03eb3313432d4d388193bdb8cada9563 (127.26.78.253:44617), 615028f6eaf44d9981fd10aa6616a579 (127.26.78.252:43529)
I20260929 19:08:08.360139 8026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b88f21361924a7fa119d04c96f67a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615028f6eaf44d9981fd10aa6616a579"
I20260929 19:08:08.360216 8026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:08.360989 8026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b88f21361924a7fa119d04c96f67a88 in term 1.
I20260929 19:08:08.361146 7951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b88f21361924a7fa119d04c96f67a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03eb3313432d4d388193bdb8cada9563"
I20260929 19:08:08.361227 7951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:08.361877 7851 leader_election.cc:302] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b88f21361924a7fa119d04c96f67a88, 615028f6eaf44d9981fd10aa6616a579; no voters:
I20260929 19:08:08.362262 7951 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b88f21361924a7fa119d04c96f67a88 in term 1.
I20260929 19:08:08.362459 8073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:08.362629 8073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [term 1 LEADER]: Becoming Leader. State: Replica: 2b88f21361924a7fa119d04c96f67a88, State: Running, Role: LEADER
I20260929 19:08:08.362728 8073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } }
I20260929 19:08:08.363111 8073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b88f21361924a7fa119d04c96f67a88. Latest consensus state: current_term: 1 leader_uuid: "2b88f21361924a7fa119d04c96f67a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.363193 8073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:08.363919 8109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:08.364108 8109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:08.365641 8026 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [term 1 FOLLOWER]: Refusing update from remote peer 2b88f21361924a7fa119d04c96f67a88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:08.365990 8107 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:08.367156 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b88f21361924a7fa119d04c96f67a88. Latest consensus state: current_term: 1 leader_uuid: "2b88f21361924a7fa119d04c96f67a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.367259 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:08:08.367594 7951 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [term 1 FOLLOWER]: Refusing update from remote peer 2b88f21361924a7fa119d04c96f67a88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:08.367830 8107 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:08.369014 8094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b88f21361924a7fa119d04c96f67a88. Latest consensus state: current_term: 1 leader_uuid: "2b88f21361924a7fa119d04c96f67a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.369122 8094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:08:08.371915 8107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b88f21361924a7fa119d04c96f67a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.372006 8107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:08.372478 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b88f21361924a7fa119d04c96f67a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.372565 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:08:08.372735 8110 mvcc.cc:205] Tried to move back new op lower bound from 7334743606743187456 to 7334743606734168064. Current Snapshot: MvccSnapshot[applied={T|T < 7334743606743187456}]
I20260929 19:08:08.372840 8111 mvcc.cc:205] Tried to move back new op lower bound from 7334743606743187456 to 7334743606734168064. Current Snapshot: MvccSnapshot[applied={T|T < 7334743606743187456}]
I20260929 19:08:08.373178 8073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b88f21361924a7fa119d04c96f67a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.373275 8073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:08.373664 8094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b88f21361924a7fa119d04c96f67a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b88f21361924a7fa119d04c96f67a88" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 33161 } } peers { permanent_uuid: "03eb3313432d4d388193bdb8cada9563" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 44617 } } peers { permanent_uuid: "615028f6eaf44d9981fd10aa6616a579" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 43529 } } }
I20260929 19:08:08.373752 8094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:08:08.374011 8113 mvcc.cc:205] Tried to move back new op lower bound from 7334743606743187456 to 7334743606734168064. Current Snapshot: MvccSnapshot[applied={T|T < 7334743606743187456}]
I20260929 19:08:08.374583 8109 catalog_manager.cc:1347] Generated new cluster ID: 2a39b54e52d64242b11b6644be300795
I20260929 19:08:08.374634 8109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:08:08.397665 8109 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:08:08.398449 8109 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:08.410277 8109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b88f21361924a7fa119d04c96f67a88: Generated new TSK 0
I20260929 19:08:08.410491 8109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:08:09.288900 8071 catalog_manager.cc:1259] Loaded cluster ID: 2a39b54e52d64242b11b6644be300795
I20260929 19:08:09.288970 8071 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: loading cluster ID for follower catalog manager: success
I20260929 19:08:09.289806 8071 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: acquiring CA information for follower catalog manager: success
I20260929 19:08:09.290182 8071 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 615028f6eaf44d9981fd10aa6616a579: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:08:09.343044 8105 catalog_manager.cc:1259] Loaded cluster ID: 2a39b54e52d64242b11b6644be300795
I20260929 19:08:09.343119 8105 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: loading cluster ID for follower catalog manager: success
I20260929 19:08:09.343876 8105 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: acquiring CA information for follower catalog manager: success
I20260929 19:08:09.344125 8105 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 03eb3313432d4d388193bdb8cada9563: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:08:09.461493 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7834
I20260929 19:08:09.466606 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7909
I20260929 19:08:09.471863 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 7984
2026-09-29T19:08:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2837 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:08:09 dist-test-slave-xz51 krb5kdc[8125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:08:09 dist-test-slave-xz51 krb5kdc[8125](info): set up 2 sockets
Sep 29 19:08:09 dist-test-slave-xz51 krb5kdc[8125](info): commencing operation
krb5kdc: starting...
W20260929 19:08:10.510085 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:08:10 dist-test-slave-xz51 krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:08:10.629525 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:39117
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:39117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:10.711581 8138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:10.711803 8138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:10.711826 8138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:10.713014 8138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:10.713060 8138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:10.713075 8138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:10.713088 8138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:10.713100 8138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:10.714746 8138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:39117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:10.715013 8138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:10.715251 8138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:10.720252 8143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:10.720248 8146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:10.720295 8144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:10.720624 8138 server_base.cc:1034] running on GCE node
I20260929 19:08:10.720886 8138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:10.721177 8138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:10.722343 8138 hybrid_clock.cc:648] HybridClock initialized: now 1790708890722328 us; error 25 us; skew 500 ppm
Sep 29 19:08:10 dist-test-slave-xz51 krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708890, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:10.725028 8138 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:10.725561 8138 webserver.cc:460] Webserver started at http://127.26.78.254:33475/ using document root <none> and password file <none>
I20260929 19:08:10.725808 8138 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:10.725942 8138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:10.726114 8138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:10.727259 8138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "2324ecb1790a4584ab0bb9dad1070ec7"
format_stamp: "Formatted at 2026-09-29 19:08:10 on dist-test-slave-xz51"
server_key: "5d7e9cae317c471cf0c6a8a72c16af8c"
server_key_iv: "7ff4adfe7485bef424195f7f02b1f592"
server_key_version: "encryptionkey@0"
I20260929 19:08:10.727666 8138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "2324ecb1790a4584ab0bb9dad1070ec7"
format_stamp: "Formatted at 2026-09-29 19:08:10 on dist-test-slave-xz51"
server_key: "5d7e9cae317c471cf0c6a8a72c16af8c"
server_key_iv: "7ff4adfe7485bef424195f7f02b1f592"
server_key_version: "encryptionkey@0"
I20260929 19:08:10.729090 8138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:08:10.729940 8153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:10.730150 8138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:10.730258 8138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "2324ecb1790a4584ab0bb9dad1070ec7"
format_stamp: "Formatted at 2026-09-29 19:08:10 on dist-test-slave-xz51"
server_key: "5d7e9cae317c471cf0c6a8a72c16af8c"
server_key_iv: "7ff4adfe7485bef424195f7f02b1f592"
server_key_version: "encryptionkey@0"
I20260929 19:08:10.730345 8138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:10.747552 8138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:10.750147 8138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:10.750363 8138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:10.754794 8138 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:39117
I20260929 19:08:10.754829 8205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:39117 every 8 connection(s)
I20260929 19:08:10.755239 8138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:10.756145 8206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:10.758778 8206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7: Bootstrap starting.
I20260929 19:08:10.759325 8206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:10.759559 8206 log.cc:824] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:10.760286 8206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7: No bootstrap required, opened a new log
I20260929 19:08:10.762012 8206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39117 } }
I20260929 19:08:10.762151 8206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:10.762177 8206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2324ecb1790a4584ab0bb9dad1070ec7, State: Initialized, Role: FOLLOWER
I20260929 19:08:10.762319 8206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39117 } }
I20260929 19:08:10.762408 8206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:10.762434 8206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:10.762462 8206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:10.762948 8206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39117 } }
I20260929 19:08:10.763042 8206 leader_election.cc:302] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2324ecb1790a4584ab0bb9dad1070ec7; no voters:
I20260929 19:08:10.763164 8206 leader_election.cc:288] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:08:10.763221 8209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:10.763429 8209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [term 1 LEADER]: Becoming Leader. State: Replica: 2324ecb1790a4584ab0bb9dad1070ec7, State: Running, Role: LEADER
I20260929 19:08:10.763494 8206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:10.763528 8209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39117 } }
I20260929 19:08:10.764262 8211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2324ecb1790a4584ab0bb9dad1070ec7. Latest consensus state: current_term: 1 leader_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39117 } } }
I20260929 19:08:10.764353 8211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:10.764519 8210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2324ecb1790a4584ab0bb9dad1070ec7" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 39117 } } }
I20260929 19:08:10.764617 8210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:10.764580 8219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:10.764715 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 8138
I20260929 19:08:10.764828 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:08:10.765039 26939 external_mini_cluster.cc:1428] Setting key 7754b6841b566d36daec828d063c85a6
I20260929 19:08:10.765286 8219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:10.766819 8219 catalog_manager.cc:1347] Generated new cluster ID: 0447f56d9ef44362b9f921b26c38a082
I20260929 19:08:10.766871 8219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:08:10 dist-test-slave-xz51 krb5kdc[8125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:10.775664 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:10.765660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.26.78.254:39117)
0929 19:08:10.765855 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:10.765865 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:10.766155 (+ 290us) server_negotiation.cc:407] Connection header received
0929 19:08:10.766429 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:10.766437 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:10.766520 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:10.766674 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:10.767385 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:10.768298 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:10.769083 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:10.769547 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:10.769673 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:10.772187 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:10.772191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:10.772192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:10.772205 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:10.774018 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:10.774410 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:10.774414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:10.774414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:10.774520 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:10.774760 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:10.774761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:10.774762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:10.775025 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:10.775200 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:10.775298 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 19:08:10.775449 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260929 19:08:10.776678 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:08:10.789283 8219 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:08:10.789940 8219 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:10.794664 8219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2324ecb1790a4584ab0bb9dad1070ec7: Generated new TSK 0
I20260929 19:08:10.794893 8219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:08:10.830762 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 8138
2026-09-29T19:08:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1357 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:08:10 dist-test-slave-xz51 krb5kdc[8233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:08:10 dist-test-slave-xz51 krb5kdc[8233](info): set up 2 sockets
Sep 29 19:08:10 dist-test-slave-xz51 krb5kdc[8233](info): commencing operation
krb5kdc: starting...
W20260929 19:08:11.870172 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:08:11 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:08:11.987089 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36761
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:36761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:12.069190 8246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:12.069443 8246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:12.069482 8246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:12.070703 8246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:12.070771 8246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:12.070796 8246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:12.070837 8246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:12.070859 8246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:12.072342 8246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:36761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:12.072626 8246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:12.072867 8246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:12.077768 8252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.077792 8251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.077797 8254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:12.077893 8246 server_base.cc:1034] running on GCE node
I20260929 19:08:12.078287 8246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:12.078550 8246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:12.079695 8246 hybrid_clock.cc:648] HybridClock initialized: now 1790708892079684 us; error 29 us; skew 500 ppm
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:12.082376 8246 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:12.082820 8246 webserver.cc:460] Webserver started at http://127.26.78.254:39889/ using document root <none> and password file <none>
I20260929 19:08:12.083061 8246 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:12.083106 8246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:12.083279 8246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:12.084367 8246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "d3094af0a2784132b5c61ac521c4cb85"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "06d5f2ed18de5d5d5aeb49509f764bc7"
server_key_iv: "ced5729effb4acf0f3aad4e4e65d743b"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.084730 8246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "d3094af0a2784132b5c61ac521c4cb85"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "06d5f2ed18de5d5d5aeb49509f764bc7"
server_key_iv: "ced5729effb4acf0f3aad4e4e65d743b"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.086123 8246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:08:12.086930 8261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.087124 8246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:12.087235 8246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "d3094af0a2784132b5c61ac521c4cb85"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "06d5f2ed18de5d5d5aeb49509f764bc7"
server_key_iv: "ced5729effb4acf0f3aad4e4e65d743b"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.087312 8246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:12.105183 8246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:12.107633 8246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:12.107805 8246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:12.112128 8246 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:36761
I20260929 19:08:12.112216 8313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:36761 every 8 connection(s)
I20260929 19:08:12.112558 8246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:12.113597 8314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:12.116046 8314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Bootstrap starting.
I20260929 19:08:12.116686 8314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:12.116999 8314 log.cc:824] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:12.117762 8314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: No bootstrap required, opened a new log
I20260929 19:08:12.119489 8314 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.119725 8314 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:12.119791 8314 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3094af0a2784132b5c61ac521c4cb85, State: Initialized, Role: FOLLOWER
I20260929 19:08:12.119944 8314 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.120030 8314 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:12.120069 8314 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:12.120121 8314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:12.120935 8314 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.121060 8314 leader_election.cc:302] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3094af0a2784132b5c61ac521c4cb85; no voters:
I20260929 19:08:12.121264 8314 leader_election.cc:288] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:08:12.121294 8317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:12.121479 8317 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 LEADER]: Becoming Leader. State: Replica: d3094af0a2784132b5c61ac521c4cb85, State: Running, Role: LEADER
I20260929 19:08:12.121531 8314 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:12.121604 8317 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.122139 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 8246
I20260929 19:08:12.122253 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:08:12.122231 8318 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3094af0a2784132b5c61ac521c4cb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } } }
I20260929 19:08:12.122243 8319 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3094af0a2784132b5c61ac521c4cb85. Latest consensus state: current_term: 1 leader_uuid: "d3094af0a2784132b5c61ac521c4cb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } } }
I20260929 19:08:12.122324 8318 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:12.122328 8319 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:12.122510 26939 external_mini_cluster.cc:1428] Setting key 2cffd8c732f4777770c1637ab55c61ed
I20260929 19:08:12.122597 8326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:12.123137 8326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:12.124948 8326 catalog_manager.cc:1347] Generated new cluster ID: 0ab5559697564df38f185229dbceda02
I20260929 19:08:12.125000 8326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:12.135059 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.123150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.26.78.254:36761)
0929 19:08:12.123490 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.123500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.123537 (+ 37us) server_negotiation.cc:407] Connection header received
0929 19:08:12.123937 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.123947 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.124012 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.124158 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:12.124970 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.126406 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.127552 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.128180 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.128279 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.131172 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:12.131176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:12.131177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:12.131190 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:12.133317 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.133721 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.133725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.133725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.133889 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.134100 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.134101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.134101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.134425 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:12.134587 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.134671 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.134819 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":88,"threads_started":1}
I20260929 19:08:12.148192 8326 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:08:12.148983 8326 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:12.154968 8326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Generated new TSK 0
I20260929 19:08:12.155220 8326 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:08:12.189007 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36761
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:12.271744 8340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:12.271960 8340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:12.271984 8340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:12.273100 8340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:12.273136 8340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:12.273164 8340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:08:12.274729 8340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:12.274937 8340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:12.275122 8340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:12.280108 8346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.280139 8348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.280342 8345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:12.280561 8340 server_base.cc:1034] running on GCE node
I20260929 19:08:12.280793 8340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:12.281070 8340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:12.282224 8340 hybrid_clock.cc:648] HybridClock initialized: now 1790708892282200 us; error 40 us; skew 500 ppm
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:12.284771 8340 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:08:12.285220 8340 webserver.cc:460] Webserver started at http://127.26.78.193:33245/ using document root <none> and password file <none>
I20260929 19:08:12.285473 8340 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:12.285544 8340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:12.285691 8340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:12.286809 8340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "63d1ce8360324439966b7a8749dbdae4"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "68388f5ae60f6aca14b9ed9833b892d6"
server_key_iv: "030a4a5c47bbcad9286c5af60c98ae52"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.287171 8340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "63d1ce8360324439966b7a8749dbdae4"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "68388f5ae60f6aca14b9ed9833b892d6"
server_key_iv: "030a4a5c47bbcad9286c5af60c98ae52"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.288524 8340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:08:12.289287 8355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.289482 8340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:12.289582 8340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "63d1ce8360324439966b7a8749dbdae4"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "68388f5ae60f6aca14b9ed9833b892d6"
server_key_iv: "030a4a5c47bbcad9286c5af60c98ae52"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.289669 8340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:12.309588 8340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:12.311600 8340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:12.311769 8340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:12.312019 8340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:12.312340 8340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:12.312402 8340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.312443 8340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:12.312471 8340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.318574 8340 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:34779
I20260929 19:08:12.318629 8468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:34779 every 8 connection(s)
I20260929 19:08:12.318993 8340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:08:12.324338 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 8340
I20260929 19:08:12.324486 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:08:12.324715 26939 external_mini_cluster.cc:1428] Setting key 4212a570cc2540e03e93c7b21992b8fc
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:12.329787 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.319775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:54499 (local address 127.26.78.254:36761)
0929 19:08:12.319840 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.319842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.320088 (+ 246us) server_negotiation.cc:407] Connection header received
0929 19:08:12.320612 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.320613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.320627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.320687 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:12.321854 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.322471 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.323522 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.323722 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.323774 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.326581 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:12.326582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:12.326583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:12.326590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:12.328263 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.328771 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.328772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.328773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.328824 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.329106 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.329107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.329107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.329210 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:12.329260 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.329643 (+ 383us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.329720 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:08:12.330098 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.319764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:54499)
0929 19:08:12.319946 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.319985 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.320513 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.320743 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.320746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.321487 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:12.321735 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.321740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.322552 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.322553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.323447 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.323450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.323796 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.323797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.323916 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.324518 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:12.324544 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:12.326492 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:12.328350 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:12.328358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:12.328361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:12.328690 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:12.328890 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:12.328891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:12.328891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:12.329042 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:12.329277 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:12.329285 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:12.329465 (+ 180us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.329598 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.329796 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260929 19:08:12.330672 8469 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36761
I20260929 19:08:12.330772 8469 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:12.330984 8469 heartbeater.cc:511] Master 127.26.78.254:36761 requested a full tablet report, sending...
I20260929 19:08:12.331542 8278 ts_manager.cc:194] Registered new tserver with Master: 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193:34779)
I20260929 19:08:12.332170 8278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:54499
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:08:12.353369 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36761
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:12.435280 8474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:12.435499 8474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:12.435523 8474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:12.436612 8474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:12.436647 8474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:12.436672 8474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:08:12.438067 8474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:12.438266 8474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:12.438457 8474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:12.443437 8482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.443440 8479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.443526 8480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:12.443583 8474 server_base.cc:1034] running on GCE node
I20260929 19:08:12.443958 8474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:12.444208 8474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:12.445379 8474 hybrid_clock.cc:648] HybridClock initialized: now 1790708892445363 us; error 44 us; skew 500 ppm
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:12.448033 8474 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:08:12.448486 8474 webserver.cc:460] Webserver started at http://127.26.78.194:39763/ using document root <none> and password file <none>
I20260929 19:08:12.448671 8474 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:12.448710 8474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:12.448798 8474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:12.449949 8474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "88d227551b3a4df4bcdd8426f8391311"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "1e188241ee4fdbb05243bf88ffcfcc07"
server_key_iv: "4027cbcca80ef50a01aed354417b642f"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.450265 8474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "88d227551b3a4df4bcdd8426f8391311"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "1e188241ee4fdbb05243bf88ffcfcc07"
server_key_iv: "4027cbcca80ef50a01aed354417b642f"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.451557 8474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:12.452261 8489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.452464 8474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:08:12.452551 8474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "88d227551b3a4df4bcdd8426f8391311"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "1e188241ee4fdbb05243bf88ffcfcc07"
server_key_iv: "4027cbcca80ef50a01aed354417b642f"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.452607 8474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:12.467041 8474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:12.468945 8474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:12.469059 8474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:12.469259 8474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:12.469590 8474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:12.469625 8474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.469669 8474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:12.469696 8474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.475520 8474 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39335
I20260929 19:08:12.475595 8602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39335 every 8 connection(s)
I20260929 19:08:12.475914 8474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:08:12.478314 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 8474
I20260929 19:08:12.478426 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:08:12.478649 26939 external_mini_cluster.cc:1428] Setting key 3432a86bc465f19a786995a2d5e5e62d
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:12.487274 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.476671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:50583 (local address 127.26.78.254:36761)
0929 19:08:12.476745 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.476747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.476961 (+ 214us) server_negotiation.cc:407] Connection header received
0929 19:08:12.477490 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.477491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.477503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.477561 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:12.478659 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.479296 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.481068 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.481424 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.481508 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.484078 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:12.484080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:12.484081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:12.484090 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:12.485809 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.486234 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.486236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.486236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.486305 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.486548 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.486549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.486549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.486704 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:12.486779 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.487045 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.487140 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:08:12.487289 8605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.476666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:50583)
0929 19:08:12.476836 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.476879 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.477385 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.477641 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.477645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.478293 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:12.478560 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.478571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.479415 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.479416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.480948 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.480952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.481528 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.481529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.481632 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.482235 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:12.482245 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:12.483975 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:12.485922 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:12.485929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:12.485931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:12.486152 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:12.486380 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:12.486380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:12.486381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:12.486478 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:12.486773 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:12.486777 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:12.486908 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.486992 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.487117 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260929 19:08:12.487870 8603 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36761
I20260929 19:08:12.487967 8603 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
I20260929 19:08:12.488183 8603 heartbeater.cc:511] Master 127.26.78.254:36761 requested a full tablet report, sending...
I20260929 19:08:12.488596 8278 ts_manager.cc:194] Registered new tserver with Master: 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335)
I20260929 19:08:12.488978 8278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:50583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:08:12.506736 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36761
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:12.588677 8608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:12.588939 8608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:12.588976 8608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:12.590233 8608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:12.590310 8608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:12.590363 8608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:08:12.591894 8608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:12.592182 8608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:12.592428 8608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:12.597672 8613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.597666 8616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.597666 8614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:12.597894 8608 server_base.cc:1034] running on GCE node
I20260929 19:08:12.598156 8608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:12.598407 8608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:12.599571 8608 hybrid_clock.cc:648] HybridClock initialized: now 1790708892599557 us; error 42 us; skew 500 ppm
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:12.602083 8608 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:08:12.602550 8608 webserver.cc:460] Webserver started at http://127.26.78.195:35149/ using document root <none> and password file <none>
I20260929 19:08:12.602732 8608 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:12.602772 8608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:12.602860 8608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:12.603863 8608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "5cb074e7d3774637b9d35e016dcfd732"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "c154dd49fe64088f9383acf349d29b7c"
server_key_iv: "45701c30b0a5f8a320cd813463a6ccbd"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.604158 8608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "5cb074e7d3774637b9d35e016dcfd732"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "c154dd49fe64088f9383acf349d29b7c"
server_key_iv: "45701c30b0a5f8a320cd813463a6ccbd"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.605453 8608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:08:12.606221 8623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.606400 8608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.606488 8608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "5cb074e7d3774637b9d35e016dcfd732"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "c154dd49fe64088f9383acf349d29b7c"
server_key_iv: "45701c30b0a5f8a320cd813463a6ccbd"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.606544 8608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:12.619901 8608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:12.621908 8608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:12.622046 8608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:12.622349 8608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:12.622639 8608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:12.622671 8608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.622692 8608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:12.622705 8608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.628526 8608 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:46595
I20260929 19:08:12.628607 8736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:46595 every 8 connection(s)
I20260929 19:08:12.628938 8608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:08:12.632015 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 8608
I20260929 19:08:12.632124 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:08:12.632316 26939 external_mini_cluster.cc:1428] Setting key eb7ef763d44e22a5b9a986d963f8b156
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:12.638304 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.629661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:33601 (local address 127.26.78.254:36761)
0929 19:08:12.629741 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.629743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.629984 (+ 241us) server_negotiation.cc:407] Connection header received
0929 19:08:12.630451 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.630452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.630462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.630516 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:12.631443 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.631933 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.633436 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.633627 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.633682 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.635873 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:12.635874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:12.635874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:12.635880 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:12.637067 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.637430 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.637431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.637431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.637479 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.637701 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.637702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.637702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.637802 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:12.637941 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.638173 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.638237 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 19:08:12.638494 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.629620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:33601)
0929 19:08:12.629797 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.629887 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.630371 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.630570 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.630573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.631153 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:12.631351 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.631356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.632053 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.632054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.633366 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.633369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.633703 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.633703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.633802 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.634429 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:12.634437 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:12.635786 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:12.637138 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:12.637143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:12.637146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:12.637365 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:12.637535 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:12.637536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:12.637536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:12.637630 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:12.637934 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:12.637939 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:12.638057 (+ 118us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.638145 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.638266 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260929 19:08:12.638949 8737 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36761
I20260929 19:08:12.639031 8737 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:12.639176 8737 heartbeater.cc:511] Master 127.26.78.254:36761 requested a full tablet report, sending...
I20260929 19:08:12.639487 8278 ts_manager.cc:194] Registered new tserver with Master: 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
I20260929 19:08:12.639881 8278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:33601
I20260929 19:08:12.644030 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:08:12.651134 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.645353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.26.78.254:36761)
0929 19:08:12.645430 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.645432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.645521 (+ 89us) server_negotiation.cc:407] Connection header received
0929 19:08:12.645556 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.645556 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.645565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.645612 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:12.646229 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.646688 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.647399 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.647592 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.647713 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.648876 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:12.648877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:12.648878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:12.648884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:12.650102 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.650409 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.650410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.650410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.650461 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:12.650625 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:12.650626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:12.650626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:12.650760 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:12.650834 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.650954 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.651015 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 19:08:12.652694 8278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:08:12.654357 8278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:08:12.661480 8753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.657092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59030 (local address 127.26.78.193:34779)
0929 19:08:12.657313 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.657316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.657366 (+ 50us) server_negotiation.cc:407] Connection header received
0929 19:08:12.657493 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.657496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.657510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.657587 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:12.657734 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.658921 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.660404 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.661087 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.661157 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.661211 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.661361 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.661406 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260929 19:08:12.661484 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.656821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:34779 (local address 127.0.0.1:59030)
0929 19:08:12.657313 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.657323 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.657370 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.657586 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.657586 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.657589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:12.657645 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.657647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.659037 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.659038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.660312 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.660315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.661174 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.661176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.661255 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.661260 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.661329 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.661379 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":28,"thread_start_us":56,"threads_started":1}
I20260929 19:08:12.661564 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.657072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.26.78.195:46595)
0929 19:08:12.657282 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.657288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.657791 (+ 503us) server_negotiation.cc:407] Connection header received
0929 19:08:12.657870 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.657872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.657888 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.657968 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:12.658134 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.659122 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.660108 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.660886 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.660957 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.661015 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.661426 (+ 411us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.661471 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260929 19:08:12.661386 8750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.656821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:46595 (local address 127.0.0.1:41968)
0929 19:08:12.657678 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.657688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.657735 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.657990 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.657991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.657993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:12.658041 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.658043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.659221 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.659222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.660000 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.660002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.661052 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.661053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.661137 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.661153 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.661218 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.661274 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":3,"thread_start_us":58,"threads_started":1}
I20260929 19:08:12.662024 8751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.656921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.26.78.194:39335)
0929 19:08:12.657287 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.657293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.657305 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:08:12.657362 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.657364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.657381 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.657466 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:12.657740 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.658720 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.660690 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.661379 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.661445 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.661502 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.661867 (+ 365us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.661924 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":74,"threads_started":1}
I20260929 19:08:12.662217 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.656813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39335 (local address 127.0.0.1:49598)
0929 19:08:12.657092 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.657140 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.657208 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.657550 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.657554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.657557 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:12.657639 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.657642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.658834 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.658835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.660588 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.660591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.661540 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.661541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.661612 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.661617 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.661670 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.662152 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":78,"thread_start_us":79,"threads_started":1}
I20260929 19:08:12.662700 8671 tablet_service.cc:1467] Processing CreateTablet for tablet 220525c4b2014b81b6243b524f794782 (DEFAULT_TABLE table=test-table [id=050f7f4c43fa44d894ce560b423c491f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:12.662822 8403 tablet_service.cc:1467] Processing CreateTablet for tablet 220525c4b2014b81b6243b524f794782 (DEFAULT_TABLE table=test-table [id=050f7f4c43fa44d894ce560b423c491f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:12.662835 8537 tablet_service.cc:1467] Processing CreateTablet for tablet 220525c4b2014b81b6243b524f794782 (DEFAULT_TABLE table=test-table [id=050f7f4c43fa44d894ce560b423c491f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:12.663087 8403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220525c4b2014b81b6243b524f794782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:12.663087 8671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220525c4b2014b81b6243b524f794782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:12.663097 8537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220525c4b2014b81b6243b524f794782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:12.665800 8754 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Bootstrap starting.
I20260929 19:08:12.666507 8754 tablet_bootstrap.cc:654] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:12.666605 8755 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Bootstrap starting.
I20260929 19:08:12.667022 8756 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Bootstrap starting.
I20260929 19:08:12.667128 8754 log.cc:824] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:12.667243 8755 tablet_bootstrap.cc:654] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:12.667723 8756 tablet_bootstrap.cc:654] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:12.667862 8755 log.cc:824] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:12.667891 8754 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: No bootstrap required, opened a new log
I20260929 19:08:12.667976 8754 ts_tablet_manager.cc:1397] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:08:12.668277 8756 log.cc:824] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:12.668695 8755 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: No bootstrap required, opened a new log
I20260929 19:08:12.668787 8755 ts_tablet_manager.cc:1397] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:08:12.669067 8756 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: No bootstrap required, opened a new log
I20260929 19:08:12.669134 8756 ts_tablet_manager.cc:1397] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:08:12.669934 8754 raft_consensus.cc:348] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.670126 8754 raft_consensus.cc:374] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:12.670174 8754 raft_consensus.cc:729] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d227551b3a4df4bcdd8426f8391311, State: Initialized, Role: FOLLOWER
I20260929 19:08:12.670312 8754 consensus_queue.cc:260] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.670588 8754 ts_tablet_manager.cc:1428] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 19:08:12.670722 8603 heartbeater.cc:503] Master 127.26.78.254:36761 was elected leader, sending a full tablet report...
I20260929 19:08:12.670820 8755 raft_consensus.cc:348] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.670981 8755 raft_consensus.cc:374] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:12.671020 8755 raft_consensus.cc:729] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d1ce8360324439966b7a8749dbdae4, State: Initialized, Role: FOLLOWER
I20260929 19:08:12.671123 8755 consensus_queue.cc:260] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.671432 8469 heartbeater.cc:503] Master 127.26.78.254:36761 was elected leader, sending a full tablet report...
I20260929 19:08:12.671398 8756 raft_consensus.cc:348] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.671572 8756 raft_consensus.cc:374] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:12.671614 8756 raft_consensus.cc:729] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb074e7d3774637b9d35e016dcfd732, State: Initialized, Role: FOLLOWER
I20260929 19:08:12.671416 8755 ts_tablet_manager.cc:1428] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:08:12.671747 8756 consensus_queue.cc:260] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.672034 8756 ts_tablet_manager.cc:1428] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:08:12.672180 8737 heartbeater.cc:503] Master 127.26.78.254:36761 was elected leader, sending a full tablet report...
I20260929 19:08:12.689028 8762 raft_consensus.cc:482] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:12.689149 8762 raft_consensus.cc:504] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.689587 8762 leader_election.cc:288] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193:34779), 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335)
I20260929 19:08:12.693627 8753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.689855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:42571 (local address 127.26.78.193:34779)
0929 19:08:12.689900 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.689902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.689912 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:08:12.690019 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.690020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.690029 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.690089 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:12.690266 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.691018 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.692609 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.693242 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.693309 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.693319 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.693488 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.693537 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:08:12.693630 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.689737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:34779 (local address 127.26.78.195:42571)
0929 19:08:12.689802 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.689814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.689900 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.690105 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.690106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.690108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:12.690184 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.690186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.691099 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.691100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.692503 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.692507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.693319 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.693320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.693415 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.693418 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.693471 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.693522 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":75}
I20260929 19:08:12.693689 8751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.689931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:59101 (local address 127.26.78.194:39335)
0929 19:08:12.689982 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.689984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.690103 (+ 119us) server_negotiation.cc:407] Connection header received
0929 19:08:12.690174 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.690175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.690185 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.690273 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:12.690437 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.691119 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.692609 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.693226 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.693320 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.693329 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.693574 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.693616 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:08:12.693792 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.689836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39335 (local address 127.26.78.195:59101)
0929 19:08:12.690014 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.690024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:12.690115 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:12.690283 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:12.690284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:12.690285 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:12.690358 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.690359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.691185 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.691186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.692503 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:12.692507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.693319 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.693320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:12.693403 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.693408 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:12.693471 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:08:12.693522 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":12,"thread_start_us":123,"threads_started":1}
I20260929 19:08:12.694082 8423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220525c4b2014b81b6243b524f794782" candidate_uuid: "5cb074e7d3774637b9d35e016dcfd732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d1ce8360324439966b7a8749dbdae4" is_pre_election: true
I20260929 19:08:12.694267 8423 raft_consensus.cc:2454] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cb074e7d3774637b9d35e016dcfd732 in term 0.
I20260929 19:08:12.694334 8557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220525c4b2014b81b6243b524f794782" candidate_uuid: "5cb074e7d3774637b9d35e016dcfd732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d227551b3a4df4bcdd8426f8391311" is_pre_election: true
I20260929 19:08:12.694516 8557 raft_consensus.cc:2454] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cb074e7d3774637b9d35e016dcfd732 in term 0.
I20260929 19:08:12.694557 8625 leader_election.cc:302] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb074e7d3774637b9d35e016dcfd732, 63d1ce8360324439966b7a8749dbdae4; no voters:
I20260929 19:08:12.694734 8762 raft_consensus.cc:2781] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:08:12.694797 8762 raft_consensus.cc:482] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:08:12.694815 8762 raft_consensus.cc:3037] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:12.695487 8762 raft_consensus.cc:504] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.695631 8762 leader_election.cc:288] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [CANDIDATE]: Term 1 election: Requested vote from peers 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193:34779), 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335)
I20260929 19:08:12.695762 8423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220525c4b2014b81b6243b524f794782" candidate_uuid: "5cb074e7d3774637b9d35e016dcfd732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d1ce8360324439966b7a8749dbdae4"
I20260929 19:08:12.695777 8557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220525c4b2014b81b6243b524f794782" candidate_uuid: "5cb074e7d3774637b9d35e016dcfd732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d227551b3a4df4bcdd8426f8391311"
I20260929 19:08:12.695852 8557 raft_consensus.cc:3037] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:12.695868 8423 raft_consensus.cc:3037] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:12.696574 8423 raft_consensus.cc:2454] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cb074e7d3774637b9d35e016dcfd732 in term 1.
I20260929 19:08:12.696591 8557 raft_consensus.cc:2454] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cb074e7d3774637b9d35e016dcfd732 in term 1.
I20260929 19:08:12.696780 8624 leader_election.cc:302] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb074e7d3774637b9d35e016dcfd732, 88d227551b3a4df4bcdd8426f8391311; no voters:
I20260929 19:08:12.696909 8762 raft_consensus.cc:2781] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:12.697099 8762 raft_consensus.cc:686] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 1 LEADER]: Becoming Leader. State: Replica: 5cb074e7d3774637b9d35e016dcfd732, State: Running, Role: LEADER
I20260929 19:08:12.697206 8762 consensus_queue.cc:237] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:12.698108 8277 catalog_manager.cc:5495] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "5cb074e7d3774637b9d35e016dcfd732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } health_report { overall_health: HEALTHY } } }
I20260929 19:08:12.707273 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.704773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.26.78.195:46595)
0929 19:08:12.704818 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:12.704821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:12.704861 (+ 40us) server_negotiation.cc:407] Connection header received
0929 19:08:12.704941 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:12.704942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:12.704959 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:12.705048 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:08:12.705182 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.705651 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.706480 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:12.706689 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:12.706773 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:12.706938 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:08:12.707040 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:08:12.707133 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:12.707153 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 19:08:12.707172 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:08:12.709052 8557 raft_consensus.cc:1261] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Refusing update from remote peer 5cb074e7d3774637b9d35e016dcfd732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:12.709052 8423 raft_consensus.cc:1261] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Refusing update from remote peer 5cb074e7d3774637b9d35e016dcfd732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:12.709419 8764 consensus_queue.cc:1035] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:12.709550 8762 consensus_queue.cc:1035] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:12.711789 8767 mvcc.cc:205] Tried to move back new op lower bound from 7334743624533508096 to 7334743624488656896. Current Snapshot: MvccSnapshot[applied={T|T < 7334743624533508096}]
I20260929 19:08:12.712709 8769 mvcc.cc:205] Tried to move back new op lower bound from 7334743624533508096 to 7334743624488656896. Current Snapshot: MvccSnapshot[applied={T|T < 7334743624533508096}]
I20260929 19:08:12.712888 8768 mvcc.cc:205] Tried to move back new op lower bound from 7334743624533508096 to 7334743624488656896. Current Snapshot: MvccSnapshot[applied={T|T < 7334743624533508096 or (T in {7334743624533508096})}]
I20260929 19:08:12.714373 8278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:12.714478 8278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:12.715194 8278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Sending DeleteTablet for 3 replicas of tablet 220525c4b2014b81b6243b524f794782
I20260929 19:08:12.715536 8403 tablet_service.cc:1514] Processing DeleteTablet for tablet 220525c4b2014b81b6243b524f794782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:12 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:59030
I20260929 19:08:12.715588 8671 tablet_service.cc:1514] Processing DeleteTablet for tablet 220525c4b2014b81b6243b524f794782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:12 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41968
I20260929 19:08:12.715598 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 8340
I20260929 19:08:12.716243 8537 tablet_service.cc:1514] Processing DeleteTablet for tablet 220525c4b2014b81b6243b524f794782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:12 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:49598
I20260929 19:08:12.716483 8776 tablet_replica.cc:330] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: stopping tablet replica
I20260929 19:08:12.716529 8777 tablet_replica.cc:330] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: stopping tablet replica
I20260929 19:08:12.716622 8777 raft_consensus.cc:2229] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:08:12.716708 8777 raft_consensus.cc:2258] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:12.716790 8776 raft_consensus.cc:2229] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:12.716953 8776 raft_consensus.cc:2258] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:12.717163 8777 ts_tablet_manager.cc:1905] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:12.717505 8776 ts_tablet_manager.cc:1905] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:12.719183 8777 ts_tablet_manager.cc:1918] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:12.719260 8777 log.cc:1196] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/220525c4b2014b81b6243b524f794782
I20260929 19:08:12.719467 8776 ts_tablet_manager.cc:1918] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:12.719574 8777 ts_tablet_manager.cc:1939] T 220525c4b2014b81b6243b524f794782 P 5cb074e7d3774637b9d35e016dcfd732: Deleting consensus metadata
I20260929 19:08:12.719595 8776 log.cc:1196] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/220525c4b2014b81b6243b524f794782
I20260929 19:08:12.719969 8776 ts_tablet_manager.cc:1939] T 220525c4b2014b81b6243b524f794782 P 88d227551b3a4df4bcdd8426f8391311: Deleting consensus metadata
I20260929 19:08:12.719991 8265 catalog_manager.cc:4841] TS 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595): tablet 220525c4b2014b81b6243b524f794782 (table test-table [id=050f7f4c43fa44d894ce560b423c491f]) successfully deleted
I20260929 19:08:12.720441 8262 catalog_manager.cc:4841] TS 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335): tablet 220525c4b2014b81b6243b524f794782 (table test-table [id=050f7f4c43fa44d894ce560b423c491f]) successfully deleted
W20260929 19:08:12.723028 8263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:34779 (error 108)
I20260929 19:08:12.723547 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:12.723425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:34779 (local address 127.0.0.1:59040)
0929 19:08:12.723474 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:12.723513 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260929 19:08:12.723811 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 8474
W20260929 19:08:12.724049 8263 catalog_manager.cc:4584] TS 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193:34779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 220525c4b2014b81b6243b524f794782: Network error: Client connection negotiation failed: client connection to 127.26.78.193:34779: connect: Connection refused (error 111)
I20260929 19:08:12.728978 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 8608
I20260929 19:08:12.733733 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 8246
I20260929 19:08:12.738994 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36761
--webserver_interface=127.26.78.254
--webserver_port=39889
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:36761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:12.820634 8779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:12.820864 8779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:12.820887 8779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:12.822106 8779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:12.822149 8779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:12.822165 8779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:12.822176 8779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:12.822189 8779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:12.823698 8779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:36761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=39889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:12.823976 8779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:12.824224 8779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:12.833001 8785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:12.833067 8779 server_base.cc:1034] running on GCE node
W20260929 19:08:12.833001 8788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.833003 8786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:12.833510 8779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:12.833946 8779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:12.835112 8779 hybrid_clock.cc:648] HybridClock initialized: now 1790708892835098 us; error 46 us; skew 500 ppm
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:12.837582 8779 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:12.838138 8779 webserver.cc:460] Webserver started at http://127.26.78.254:39889/ using document root <none> and password file <none>
I20260929 19:08:12.838390 8779 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:12.838478 8779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:12.839850 8779 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:12.840613 8795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.840807 8779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:12.840931 8779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "d3094af0a2784132b5c61ac521c4cb85"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "06d5f2ed18de5d5d5aeb49509f764bc7"
server_key_iv: "ced5729effb4acf0f3aad4e4e65d743b"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.841372 8779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:12.865794 8779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:12.867754 8779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:12.867938 8779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:12.872144 8779 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:36761
I20260929 19:08:12.872184 8847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:36761 every 8 connection(s)
I20260929 19:08:12.872602 8779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:12.874119 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 8779
I20260929 19:08:12.874418 8848 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:08:12.874419 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:34779
--local_ip_for_outbound_sockets=127.26.78.193
--tserver_master_addrs=127.26.78.254:36761
--webserver_port=33245
--webserver_interface=127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:12.875533 8848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Bootstrap starting.
I20260929 19:08:12.883937 8848 log.cc:824] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:12.886065 8848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:08:12.886406 8848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Bootstrap complete.
I20260929 19:08:12.888799 8848 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.889037 8848 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3094af0a2784132b5c61ac521c4cb85, State: Initialized, Role: FOLLOWER
I20260929 19:08:12.889217 8848 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.889279 8848 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:12.889335 8848 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:12.889397 8848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:08:12.890539 8848 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.890628 8848 leader_election.cc:302] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3094af0a2784132b5c61ac521c4cb85; no voters:
I20260929 19:08:12.890872 8848 leader_election.cc:288] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:08:12.890916 8852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:08:12.891032 8852 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 LEADER]: Becoming Leader. State: Replica: d3094af0a2784132b5c61ac521c4cb85, State: Running, Role: LEADER
I20260929 19:08:12.891116 8852 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:12.891145 8848 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:12.891577 8854 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3094af0a2784132b5c61ac521c4cb85. Latest consensus state: current_term: 2 leader_uuid: "d3094af0a2784132b5c61ac521c4cb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } } }
I20260929 19:08:12.891654 8854 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:12.891906 8856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:12.892328 8853 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d3094af0a2784132b5c61ac521c4cb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } } }
I20260929 19:08:12.892472 8853 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:12.892716 8856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:12.892807 8856 catalog_manager.cc:1259] Loaded cluster ID: 0ab5559697564df38f185229dbceda02
I20260929 19:08:12.892838 8856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:08:12.893181 8856 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 19:08:12.897867 8856 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 19:08:12.900424 8856 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:12.900677 8856 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Loaded TSK: 0
I20260929 19:08:12.900908 8856 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:08:12.958113 8850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:12.958353 8850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:12.958390 8850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:12.959558 8850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:12.959626 8850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:12.959671 8850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:08:12.961201 8850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:34779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=33245
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:12.961490 8850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:12.961725 8850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:12.967597 8875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.967597 8876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:12.967597 8878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:12.967900 8850 server_base.cc:1034] running on GCE node
I20260929 19:08:12.968096 8850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:12.968351 8850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:12.969513 8850 hybrid_clock.cc:648] HybridClock initialized: now 1790708892969505 us; error 36 us; skew 500 ppm
Sep 29 19:08:12 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:12.979687 8850 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:08:12.980304 8850 webserver.cc:460] Webserver started at http://127.26.78.193:33245/ using document root <none> and password file <none>
I20260929 19:08:12.980626 8850 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:12.980719 8850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:12.982581 8850 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:08:12.983362 8885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:12.983577 8850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:12.983681 8850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "63d1ce8360324439966b7a8749dbdae4"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "68388f5ae60f6aca14b9ed9833b892d6"
server_key_iv: "030a4a5c47bbcad9286c5af60c98ae52"
server_key_version: "encryptionkey@0"
I20260929 19:08:12.984123 8850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:13.008849 8850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:13.011248 8850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:13.011430 8850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:13.011719 8850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:13.012274 8892 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:08:13.013723 8850 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:08:13.013774 8850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 19:08:13.013811 8850 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:08:13.014524 8850 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:08:13.014570 8850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 19:08:13.014616 8892 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Bootstrap starting.
I20260929 19:08:13.022310 8850 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:34779
I20260929 19:08:13.022459 8999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:34779 every 8 connection(s)
I20260929 19:08:13.022841 8850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:08:13.025662 8892 log.cc:824] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:13.028062 8892 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:08:13.028366 8892 tablet_bootstrap.cc:492] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Bootstrap complete.
I20260929 19:08:13.029590 8892 ts_tablet_manager.cc:1397] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260929 19:08:13.030103 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 8850
I20260929 19:08:13.030400 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:39335
--local_ip_for_outbound_sockets=127.26.78.194
--tserver_master_addrs=127.26.78.254:36761
--webserver_port=39763
--webserver_interface=127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:13.031358 8892 raft_consensus.cc:348] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:13.031769 8892 raft_consensus.cc:729] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d1ce8360324439966b7a8749dbdae4, State: Initialized, Role: FOLLOWER
I20260929 19:08:13.031965 8892 consensus_queue.cc:260] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } } peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } }
I20260929 19:08:13.032219 8892 ts_tablet_manager.cc:1428] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Time spent starting tablet: real 0.003s user 0.003s sys 0.003s
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:13.038116 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.023549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:37035)
0929 19:08:13.025910 (+ 2361us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.025958 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.026656 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.027198 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.027203 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.027567 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:13.027831 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.027837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.029585 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.029586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.030768 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.030771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.031385 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.031386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.031529 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.032171 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:13.032182 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:13.034183 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:13.036113 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.036121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.036123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.036487 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.036797 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.036798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.036799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.036939 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.037485 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:13.037492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:13.037667 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.037770 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.037961 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2314,"thread_start_us":84,"threads_started":1}
I20260929 19:08:13.038307 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.023685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:37035 (local address 127.26.78.254:36761)
0929 19:08:13.023877 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.023887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.026078 (+ 2191us) server_negotiation.cc:407] Connection header received
0929 19:08:13.026906 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.026916 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.026978 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.027136 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:13.027976 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.029483 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.030889 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.031272 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.031427 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.034305 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:13.034309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:13.034309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:13.034319 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:13.036022 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.036584 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.036586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.036587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.036713 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.037017 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.037019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.037019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.037399 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:13.037495 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.037895 (+ 400us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.038060 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260929 19:08:13.039027 9000 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36761
I20260929 19:08:13.039132 9000 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:13.039322 9000 heartbeater.cc:511] Master 127.26.78.254:36761 requested a full tablet report, sending...
I20260929 19:08:13.040025 8812 ts_manager.cc:194] Registered new tserver with Master: 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193:34779)
I20260929 19:08:13.041179 8812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:37035
I20260929 19:08:13.042105 9000 heartbeater.cc:503] Master 127.26.78.254:36761 was elected leader, sending a full tablet report...
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:08:13.049749 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.040792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.26.78.193:34779)
0929 19:08:13.040945 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.040949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.040959 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:08:13.040991 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.040993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.041005 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.041074 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:13.041673 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.042897 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.043713 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.044231 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.044299 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.046839 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:13.046846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:13.046847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:13.046859 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:13.048452 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.048914 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.048916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.048916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.048970 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.049166 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.049167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.049167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.049302 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:13.049367 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.049552 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.049627 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260929 19:08:13.049770 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.040598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:34779 (local address 127.0.0.1:59050)
0929 19:08:13.040763 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.040807 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.040877 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.041097 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.041104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.041503 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:13.041567 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.041570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.042980 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.042981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.043609 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.043611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.044299 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.044300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.044386 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.044925 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:13.044930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:13.046759 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:13.048560 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.048568 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.048570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.048832 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.049037 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.049038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.049039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.049106 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.049373 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:13.049379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:13.049433 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.049523 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.049644 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260929 19:08:13.050230 8932 tablet_service.cc:1514] Processing DeleteTablet for tablet 220525c4b2014b81b6243b524f794782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:12 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:59050
I20260929 19:08:13.050952 9009 tablet_replica.cc:330] stopping tablet replica
I20260929 19:08:13.051080 9009 raft_consensus.cc:2229] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:13.051170 9009 raft_consensus.cc:2258] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:13.051484 9009 ts_tablet_manager.cc:1905] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:13.053553 9009 ts_tablet_manager.cc:1918] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:13.053645 9009 log.cc:1196] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/220525c4b2014b81b6243b524f794782
I20260929 19:08:13.053957 9009 ts_tablet_manager.cc:1939] T 220525c4b2014b81b6243b524f794782 P 63d1ce8360324439966b7a8749dbdae4: Deleting consensus metadata
I20260929 19:08:13.054502 8797 catalog_manager.cc:4841] TS 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193:34779): tablet 220525c4b2014b81b6243b524f794782 (table test-table [id=050f7f4c43fa44d894ce560b423c491f]) successfully deleted
W20260929 19:08:13.117568 9005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:13.117774 9005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:13.117799 9005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:13.118994 9005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:13.119037 9005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:13.119063 9005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:08:13.121812 9005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:39335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=39763
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:13.122123 9005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:13.122411 9005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:13.128988 9017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.129000 9015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.129122 9005 server_base.cc:1034] running on GCE node
W20260929 19:08:13.129052 9014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.129361 9005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:13.129575 9005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:13.130791 9005 hybrid_clock.cc:648] HybridClock initialized: now 1790708893130774 us; error 29 us; skew 500 ppm
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:13.133261 9005 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:08:13.133770 9005 webserver.cc:460] Webserver started at http://127.26.78.194:39763/ using document root <none> and password file <none>
I20260929 19:08:13.134032 9005 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:13.134117 9005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:13.135453 9005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.136085 9024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.136273 9005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.136381 9005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "88d227551b3a4df4bcdd8426f8391311"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "1e188241ee4fdbb05243bf88ffcfcc07"
server_key_iv: "4027cbcca80ef50a01aed354417b642f"
server_key_version: "encryptionkey@0"
I20260929 19:08:13.136756 9005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:13.146078 9005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:13.148015 9005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:13.148180 9005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:13.148417 9005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:13.148721 9005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:13.148777 9005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.148818 9005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:13.148846 9005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.154907 9005 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39335
I20260929 19:08:13.154958 9137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39335 every 8 connection(s)
I20260929 19:08:13.155310 9005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:08:13.156368 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9005
I20260929 19:08:13.156662 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:46595
--local_ip_for_outbound_sockets=127.26.78.195
--tserver_master_addrs=127.26.78.254:36761
--webserver_port=35149
--webserver_interface=127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:13.166520 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.156054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:53193 (local address 127.26.78.254:36761)
0929 19:08:13.156115 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.156117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.156462 (+ 345us) server_negotiation.cc:407] Connection header received
0929 19:08:13.157253 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.157254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.157264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.157310 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:13.158613 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.159390 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.160419 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.160636 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.160715 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.163471 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:13.163472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:13.163473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:13.163480 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:13.164935 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.165454 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.165455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.165455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.165507 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.165803 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.165804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.165804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.165964 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:13.166018 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.166377 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.166451 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:08:13.166863 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.156063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:53193)
0929 19:08:13.156281 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.156331 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.157125 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.157427 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.157432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.158300 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:13.158515 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.158520 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.159491 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.159492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.160302 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.160304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.160724 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.160725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.160867 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.161433 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:13.161441 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:13.163356 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:13.165020 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.165027 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.165029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.165369 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.165576 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.165576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.165577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.165734 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.166030 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:13.166037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:13.166224 (+ 187us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.166356 (+ 132us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.166552 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260929 19:08:13.167441 9138 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36761
I20260929 19:08:13.167536 9138 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:13.167735 9138 heartbeater.cc:511] Master 127.26.78.254:36761 requested a full tablet report, sending...
I20260929 19:08:13.168124 8812 ts_manager.cc:194] Registered new tserver with Master: 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335)
I20260929 19:08:13.168500 8812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:53193
W20260929 19:08:13.241550 9141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:13.241809 9141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:13.241881 9141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:13.243080 9141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:13.243153 9141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:13.243198 9141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:08:13.244973 9141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:46595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=35149
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:13.245234 9141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:13.245508 9141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:13.250716 9146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.250867 9141 server_base.cc:1034] running on GCE node
W20260929 19:08:13.250716 9149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.250759 9147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.251159 9141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:13.251449 9141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:13.252600 9141 hybrid_clock.cc:648] HybridClock initialized: now 1790708893252572 us; error 41 us; skew 500 ppm
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:13.255141 9141 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:08:13.255601 9141 webserver.cc:460] Webserver started at http://127.26.78.195:35149/ using document root <none> and password file <none>
I20260929 19:08:13.255785 9141 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:13.255838 9141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:13.257122 9141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:13.257658 9156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.257882 9141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.257954 9141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "5cb074e7d3774637b9d35e016dcfd732"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "c154dd49fe64088f9383acf349d29b7c"
server_key_iv: "45701c30b0a5f8a320cd813463a6ccbd"
server_key_version: "encryptionkey@0"
I20260929 19:08:13.258306 9141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:13.286562 9141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:13.288606 9141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:13.288758 9141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:13.289016 9141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:13.289350 9141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:13.289389 9141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.289443 9141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:13.289472 9141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.295646 9141 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:46595
I20260929 19:08:13.295686 9269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:46595 every 8 connection(s)
I20260929 19:08:13.296041 9141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:13.302572 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9141
I20260929 19:08:13.305473 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.296751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:40549 (local address 127.26.78.254:36761)
0929 19:08:13.296817 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.296819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.297104 (+ 285us) server_negotiation.cc:407] Connection header received
0929 19:08:13.297587 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.297588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.297598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.297654 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:13.298628 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.299114 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.300067 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.300247 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.300299 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.302398 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:13.302400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:13.302400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:13.302407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:13.304077 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.304541 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.304541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.304542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.304592 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.304880 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.304880 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.304880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.304977 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:13.305026 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.305345 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.305407 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:08:13.305753 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.296759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:40549)
0929 19:08:13.296945 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.296983 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.297501 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.297708 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.297711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.298342 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:13.298546 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.298551 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.299193 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.299194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.299992 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.299994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.300320 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.300320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.300427 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.300891 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:13.300899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:13.302309 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:13.304154 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.304162 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.304164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.304469 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.304669 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:13.304670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:13.304670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:13.304812 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:13.305043 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:13.305050 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:13.305209 (+ 159us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.305323 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.305489 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260929 19:08:13.306301 9270 heartbeater.cc:347] Connected to a master server at 127.26.78.254:36761
I20260929 19:08:13.306424 9270 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:13.306643 9270 heartbeater.cc:511] Master 127.26.78.254:36761 requested a full tablet report, sending...
I20260929 19:08:13.307075 8812 ts_manager.cc:194] Registered new tserver with Master: 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
I20260929 19:08:13.307403 8812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:40549
I20260929 19:08:13.309733 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.303006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49028 (local address 127.26.78.254:36761)
0929 19:08:13.303132 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.303135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.303199 (+ 64us) server_negotiation.cc:407] Connection header received
0929 19:08:13.303284 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.303285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.303296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.303361 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:13.303904 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.304589 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.305651 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.305988 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.306049 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.307539 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:13.307541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:13.307541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:13.307548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:13.308868 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.309175 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.309176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.309176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.309224 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:13.309339 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:13.309340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:13.309340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:13.309426 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:13.309531 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.309585 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.309643 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260929 19:08:13.310446 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:08:13.313508 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.310807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.26.78.254:36761)
0929 19:08:13.310876 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.310878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.311049 (+ 171us) server_negotiation.cc:407] Connection header received
0929 19:08:13.311146 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.311147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.311192 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.311280 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:08:13.311424 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.311911 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.312693 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.312908 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.312994 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.313179 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:08:13.313291 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:08:13.313389 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.313408 (+ 19us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.313430 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:08:13.313793 8812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:08:13.315461 8812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:08:13.317684 8932 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0c307e91174864a3e9fdaa884c570c (DEFAULT_TABLE table=test-table [id=04cc2c80ada2445e9f19b869f03ebac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:13.317934 8932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0c307e91174864a3e9fdaa884c570c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:13.319607 8892 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Bootstrap starting.
I20260929 19:08:13.320008 8892 tablet_bootstrap.cc:654] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:13.320564 8892 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: No bootstrap required, opened a new log
I20260929 19:08:13.320634 8892 ts_tablet_manager.cc:1397] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:13.320971 8892 raft_consensus.cc:348] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.321056 8892 raft_consensus.cc:374] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:13.321110 8892 raft_consensus.cc:729] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d1ce8360324439966b7a8749dbdae4, State: Initialized, Role: FOLLOWER
I20260929 19:08:13.321157 8892 consensus_queue.cc:260] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.321257 8892 ts_tablet_manager.cc:1428] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:13.321798 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.317421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:46595 (local address 127.0.0.1:41992)
0929 19:08:13.317618 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.317629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.317684 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.318082 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.318083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.318084 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:13.318117 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.318118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.319222 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.319223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.320476 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.320478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.321459 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.321460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.321536 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.321567 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.321655 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.321705 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":30,"thread_start_us":127,"threads_started":1}
I20260929 19:08:13.321975 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.317670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.26.78.195:46595)
0929 19:08:13.317879 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.317884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.317893 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:08:13.317930 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.317932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.317947 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.318194 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:13.318254 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.319110 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.320578 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.321369 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.321439 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.321502 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.321672 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.321717 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
I20260929 19:08:13.322388 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.317328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39335 (local address 127.0.0.1:49606)
0929 19:08:13.317422 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.317433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.317467 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.317976 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.317977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.317978 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:13.318027 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.318029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.319272 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.319272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.321133 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.321136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.322127 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.322128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.322211 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.322216 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.322277 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.322323 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":694}
I20260929 19:08:13.322420 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.317419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.26.78.194:39335)
0929 19:08:13.317734 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.317739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.317750 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:08:13.317789 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.317791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.317807 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.317966 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:13.318109 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.319138 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.321236 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.322018 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.322124 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.322182 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.322310 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.322355 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":140,"threads_started":1}
I20260929 19:08:13.322710 9204 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0c307e91174864a3e9fdaa884c570c (DEFAULT_TABLE table=test-table [id=04cc2c80ada2445e9f19b869f03ebac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:13.323138 9204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0c307e91174864a3e9fdaa884c570c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:13.323125 9072 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0c307e91174864a3e9fdaa884c570c (DEFAULT_TABLE table=test-table [id=04cc2c80ada2445e9f19b869f03ebac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:13.323485 9072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0c307e91174864a3e9fdaa884c570c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:13.326891 9281 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Bootstrap starting.
I20260929 19:08:13.327060 9280 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Bootstrap starting.
I20260929 19:08:13.327600 9281 tablet_bootstrap.cc:654] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:13.327631 9280 tablet_bootstrap.cc:654] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:13.328207 9281 log.cc:824] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:13.328222 9280 log.cc:824] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:13.329013 9281 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: No bootstrap required, opened a new log
I20260929 19:08:13.329089 9281 ts_tablet_manager.cc:1397] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:08:13.329133 9280 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: No bootstrap required, opened a new log
I20260929 19:08:13.329200 9280 ts_tablet_manager.cc:1397] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:08:13.331103 9280 raft_consensus.cc:348] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.331103 9281 raft_consensus.cc:348] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.331278 9280 raft_consensus.cc:374] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:13.331278 9281 raft_consensus.cc:374] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:13.331317 9281 raft_consensus.cc:729] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb074e7d3774637b9d35e016dcfd732, State: Initialized, Role: FOLLOWER
I20260929 19:08:13.331317 9280 raft_consensus.cc:729] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d227551b3a4df4bcdd8426f8391311, State: Initialized, Role: FOLLOWER
I20260929 19:08:13.331459 9280 consensus_queue.cc:260] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.331459 9281 consensus_queue.cc:260] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.331753 9281 ts_tablet_manager.cc:1428] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:08:13.331753 9280 ts_tablet_manager.cc:1428] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:08:13.331765 9138 heartbeater.cc:503] Master 127.26.78.254:36761 was elected leader, sending a full tablet report...
I20260929 19:08:13.331934 9270 heartbeater.cc:503] Master 127.26.78.254:36761 was elected leader, sending a full tablet report...
I20260929 19:08:13.391417 9006 raft_consensus.cc:482] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:13.391590 9006 raft_consensus.cc:504] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.392030 9006 leader_election.cc:288] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335), 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
I20260929 19:08:13.395136 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.392414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:40181 (local address 127.26.78.195:46595)
0929 19:08:13.392456 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.392458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.392513 (+ 55us) server_negotiation.cc:407] Connection header received
0929 19:08:13.392543 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.392544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.392553 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.392613 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:13.392795 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.393245 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.394396 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.394804 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.394888 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.394898 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.395031 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.395068 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:08:13.395143 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.392193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39335 (local address 127.26.78.193:50581)
0929 19:08:13.392283 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.392296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.392363 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.392539 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.392540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.392542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:13.392621 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.392624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.393413 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.393414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.394275 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.394277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.394832 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.394834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.394920 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.394943 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.395025 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.395068 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":40}
I20260929 19:08:13.395135 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.392325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:50581 (local address 127.26.78.194:39335)
0929 19:08:13.392373 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.392375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.392386 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:08:13.392444 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.392445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.392457 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.392516 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:13.392699 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.393314 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.394364 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.394750 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.394806 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.394813 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.395030 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.395066 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:08:13.395174 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.392226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:46595 (local address 127.26.78.193:40181)
0929 19:08:13.392429 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.392439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.392500 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.392626 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:13.392627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:13.392628 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:13.392724 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.392726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.393320 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.393321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.394285 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:13.394288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.394904 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.394905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:13.394952 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.394956 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:08:13.395023 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 19:08:13.395061 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":31,"thread_start_us":111,"threads_started":1}
I20260929 19:08:13.395536 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c307e91174864a3e9fdaa884c570c" candidate_uuid: "63d1ce8360324439966b7a8749dbdae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d227551b3a4df4bcdd8426f8391311" is_pre_election: true
I20260929 19:08:13.395524 9224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c307e91174864a3e9fdaa884c570c" candidate_uuid: "63d1ce8360324439966b7a8749dbdae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb074e7d3774637b9d35e016dcfd732" is_pre_election: true
I20260929 19:08:13.395695 9224 raft_consensus.cc:2454] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63d1ce8360324439966b7a8749dbdae4 in term 0.
I20260929 19:08:13.395694 9092 raft_consensus.cc:2454] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63d1ce8360324439966b7a8749dbdae4 in term 0.
I20260929 19:08:13.395937 8889 leader_election.cc:302] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb074e7d3774637b9d35e016dcfd732, 63d1ce8360324439966b7a8749dbdae4; no voters:
I20260929 19:08:13.396092 9006 raft_consensus.cc:2781] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:08:13.396179 9006 raft_consensus.cc:482] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:08:13.396219 9006 raft_consensus.cc:3037] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:13.396930 9006 raft_consensus.cc:504] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.397083 9006 leader_election.cc:288] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 1 election: Requested vote from peers 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335), 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
I20260929 19:08:13.397248 9224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c307e91174864a3e9fdaa884c570c" candidate_uuid: "63d1ce8360324439966b7a8749dbdae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb074e7d3774637b9d35e016dcfd732"
I20260929 19:08:13.397331 9224 raft_consensus.cc:3037] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:13.398121 9224 raft_consensus.cc:2454] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63d1ce8360324439966b7a8749dbdae4 in term 1.
I20260929 19:08:13.398239 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c307e91174864a3e9fdaa884c570c" candidate_uuid: "63d1ce8360324439966b7a8749dbdae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d227551b3a4df4bcdd8426f8391311"
I20260929 19:08:13.398263 8889 leader_election.cc:302] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb074e7d3774637b9d35e016dcfd732, 63d1ce8360324439966b7a8749dbdae4; no voters:
I20260929 19:08:13.398321 9092 raft_consensus.cc:3037] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:13.398368 9006 raft_consensus.cc:2781] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:13.398537 9006 raft_consensus.cc:686] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 LEADER]: Becoming Leader. State: Replica: 63d1ce8360324439966b7a8749dbdae4, State: Running, Role: LEADER
I20260929 19:08:13.398638 9006 consensus_queue.cc:237] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.399227 9092 raft_consensus.cc:2454] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63d1ce8360324439966b7a8749dbdae4 in term 1.
I20260929 19:08:13.399791 8812 catalog_manager.cc:5495] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193). New cstate: current_term: 1 leader_uuid: "63d1ce8360324439966b7a8749dbdae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } health_report { overall_health: HEALTHY } } }
W20260929 19:08:13.405884 9139 tablet.cc:2369] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:08:13.414772 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.412295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.26.78.193:34779)
0929 19:08:13.412361 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:13.412363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:13.412374 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:08:13.412414 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:13.412415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:13.412442 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:13.412520 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:08:13.412641 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.413152 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.413960 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:13.414182 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:13.414262 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:13.414421 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:08:13.414528 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:08:13.414606 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:13.414661 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 19:08:13.414686 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:08:13.416329 9092 raft_consensus.cc:1261] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Refusing update from remote peer 63d1ce8360324439966b7a8749dbdae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:13.416326 9224 raft_consensus.cc:1261] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 1 FOLLOWER]: Refusing update from remote peer 63d1ce8360324439966b7a8749dbdae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:13.416662 9006 consensus_queue.cc:1035] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:13.416846 9287 consensus_queue.cc:1035] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:13.419318 9290 mvcc.cc:205] Tried to move back new op lower bound from 7334743627430678528 to 7334743627361419264. Current Snapshot: MvccSnapshot[applied={T|T < 7334743627430678528}]
I20260929 19:08:13.421779 8812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:13.421923 8812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:13.422690 8812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Sending DeleteTablet for 3 replicas of tablet 2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:13.423035 8932 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0c307e91174864a3e9fdaa884c570c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:13 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:59050
I20260929 19:08:13.423043 9072 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0c307e91174864a3e9fdaa884c570c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:13 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:49606
I20260929 19:08:13.423058 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 8850
I20260929 19:08:13.423123 9009 tablet_replica.cc:330] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: stopping tablet replica
I20260929 19:08:13.423174 9009 raft_consensus.cc:2229] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:08:13.423247 9009 raft_consensus.cc:2258] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:13.423709 9299 tablet_replica.cc:330] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: stopping tablet replica
I20260929 19:08:13.423802 9299 raft_consensus.cc:2229] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:13.423910 9299 raft_consensus.cc:2258] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:13.424640 9204 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0c307e91174864a3e9fdaa884c570c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:13 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:41992
I20260929 19:08:13.424865 9300 tablet_replica.cc:330] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: stopping tablet replica
I20260929 19:08:13.424966 9300 raft_consensus.cc:2229] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:13.425071 9300 raft_consensus.cc:2258] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:13.425241 9299 ts_tablet_manager.cc:1905] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:13.425463 9300 ts_tablet_manager.cc:1905] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:13.426954 9299 ts_tablet_manager.cc:1918] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:13.427042 9299 log.cc:1196] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:13.427299 9299 ts_tablet_manager.cc:1939] T 2b0c307e91174864a3e9fdaa884c570c P 88d227551b3a4df4bcdd8426f8391311: Deleting consensus metadata
I20260929 19:08:13.427429 9300 ts_tablet_manager.cc:1918] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:13.427507 9300 log.cc:1196] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:13.427767 8796 catalog_manager.cc:4841] TS 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335): tablet 2b0c307e91174864a3e9fdaa884c570c (table test-table [id=04cc2c80ada2445e9f19b869f03ebac9]) successfully deleted
I20260929 19:08:13.427802 9300 ts_tablet_manager.cc:1939] T 2b0c307e91174864a3e9fdaa884c570c P 5cb074e7d3774637b9d35e016dcfd732: Deleting consensus metadata
I20260929 19:08:13.428385 8799 catalog_manager.cc:4841] TS 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595): tablet 2b0c307e91174864a3e9fdaa884c570c (table test-table [id=04cc2c80ada2445e9f19b869f03ebac9]) successfully deleted
W20260929 19:08:13.430258 8797 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:34779 (error 108)
I20260929 19:08:13.430817 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.430658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:34779 (local address 127.0.0.1:59070)
0929 19:08:13.430721 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.430757 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 19:08:13.430902 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 9005
W20260929 19:08:13.430974 8797 catalog_manager.cc:4584] TS 63d1ce8360324439966b7a8749dbdae4 (127.26.78.193:34779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b0c307e91174864a3e9fdaa884c570c: Network error: Client connection negotiation failed: client connection to 127.26.78.193:34779: connect: Connection refused (error 111)
I20260929 19:08:13.436280 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 9141
I20260929 19:08:13.440757 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 8779
I20260929 19:08:13.446283 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36761
--webserver_interface=127.26.78.254
--webserver_port=39889
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:36761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:13.526439 9302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:13.526659 9302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:13.526681 9302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:13.527863 9302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:13.527905 9302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:13.527920 9302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:13.527964 9302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:13.527995 9302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:13.529440 9302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:36761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:36761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=39889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:13.529702 9302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:13.529966 9302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:13.534746 9307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.534754 9308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.534754 9310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.534982 9302 server_base.cc:1034] running on GCE node
I20260929 19:08:13.535251 9302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:13.535534 9302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:13.536701 9302 hybrid_clock.cc:648] HybridClock initialized: now 1790708893536687 us; error 27 us; skew 500 ppm
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:13.539218 9302 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:13.539646 9302 webserver.cc:460] Webserver started at http://127.26.78.254:39889/ using document root <none> and password file <none>
I20260929 19:08:13.539885 9302 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:13.539954 9302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:13.541301 9302 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.542018 9317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.542191 9302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.542295 9302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "d3094af0a2784132b5c61ac521c4cb85"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "06d5f2ed18de5d5d5aeb49509f764bc7"
server_key_iv: "ced5729effb4acf0f3aad4e4e65d743b"
server_key_version: "encryptionkey@0"
I20260929 19:08:13.542645 9302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:13.550483 9302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:13.552788 9302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:13.552958 9302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:13.556979 9302 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:36761
I20260929 19:08:13.557013 9369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:36761 every 8 connection(s)
I20260929 19:08:13.557394 9302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:13.559018 9370 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:08:13.559607 9370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Bootstrap starting.
I20260929 19:08:13.561057 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9302
I20260929 19:08:13.561419 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:34779
--local_ip_for_outbound_sockets=127.26.78.193
--tserver_master_addrs=127.26.78.254:36761
--webserver_port=33245
--webserver_interface=127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:13.568423 9370 log.cc:824] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:13.570770 9370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:08:13.571102 9370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85: Bootstrap complete.
I20260929 19:08:13.573341 9370 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:13.573566 9370 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3094af0a2784132b5c61ac521c4cb85, State: Initialized, Role: FOLLOWER
I20260929 19:08:13.573743 9370 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:13.573803 9370 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:13.573899 9370 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:13.573944 9370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 2 FOLLOWER]: Advancing to term 3
I20260929 19:08:13.574944 9370 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:13.575055 9370 leader_election.cc:302] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3094af0a2784132b5c61ac521c4cb85; no voters:
I20260929 19:08:13.575276 9370 leader_election.cc:288] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 19:08:13.575314 9374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 19:08:13.575438 9374 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [term 3 LEADER]: Becoming Leader. State: Replica: d3094af0a2784132b5c61ac521c4cb85, State: Running, Role: LEADER
I20260929 19:08:13.575546 9370 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:13.575523 9374 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } }
I20260929 19:08:13.575809 9374 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3094af0a2784132b5c61ac521c4cb85. Latest consensus state: current_term: 3 leader_uuid: "d3094af0a2784132b5c61ac521c4cb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } } }
I20260929 19:08:13.575870 9374 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:13.576086 9375 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d3094af0a2784132b5c61ac521c4cb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3094af0a2784132b5c61ac521c4cb85" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 36761 } } }
I20260929 19:08:13.576184 9375 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3094af0a2784132b5c61ac521c4cb85 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:13.576268 9378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:13.577052 9378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:13.577154 9378 catalog_manager.cc:1259] Loaded cluster ID: 0ab5559697564df38f185229dbceda02
I20260929 19:08:13.577625 9378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 19:08:13.577904 9378 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f289085e9ad google::LogMessage::Fail() at ??:0
@ 0x7f289086188a google::LogMessage::SendToLog() at ??:0
@ 0x7f289085e4de google::LogMessage::Flush() at ??:0
@ 0x7f289085fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f289877f0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f28987888b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f289272c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2892721954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f28935856db start_thread at ??:0
@ 0x7f288f35c61f clone at ??:0
W20260929 19:08:13.661283 9372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:13.661491 9372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:13.661517 9372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:13.662766 9372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:13.662840 9372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:13.662889 9372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:08:13.664407 9372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:34779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=33245
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:13.664654 9372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:13.664904 9372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:13.670223 9409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.670231 9407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.670230 9406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.671258 9372 server_base.cc:1034] running on GCE node
I20260929 19:08:13.671545 9372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:13.671832 9372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:13.673025 9372 hybrid_clock.cc:648] HybridClock initialized: now 1790708893672991 us; error 53 us; skew 500 ppm
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:13.675916 9372 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:08:13.676414 9372 webserver.cc:460] Webserver started at http://127.26.78.193:33245/ using document root <none> and password file <none>
I20260929 19:08:13.676658 9372 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:13.676728 9372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:13.678153 9372 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.678869 9416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.679064 9372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.679171 9372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "63d1ce8360324439966b7a8749dbdae4"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "68388f5ae60f6aca14b9ed9833b892d6"
server_key_iv: "030a4a5c47bbcad9286c5af60c98ae52"
server_key_version: "encryptionkey@0"
I20260929 19:08:13.679530 9372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:13.696144 9372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:13.698715 9372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:13.698907 9372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:13.699158 9372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:13.699620 9423 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:08:13.701639 9372 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:08:13.701705 9372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 19:08:13.701772 9372 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:08:13.702461 9372 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:08:13.702520 9372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 19:08:13.702587 9423 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Bootstrap starting.
I20260929 19:08:13.709749 9372 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:34779
I20260929 19:08:13.710165 9372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:08:13.713641 9423 log.cc:824] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:13.713984 9530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:34779 every 8 connection(s)
I20260929 19:08:13.716368 9423 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:08:13.716677 9423 tablet_bootstrap.cc:492] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Bootstrap complete.
I20260929 19:08:13.717322 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9372
I20260929 19:08:13.717602 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:39335
--local_ip_for_outbound_sockets=127.26.78.194
--tserver_master_addrs=127.26.78.254:36761
--webserver_port=39763
--webserver_interface=127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:13.717933 9423 ts_tablet_manager.cc:1397] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.015s
I20260929 19:08:13.719417 9423 raft_consensus.cc:348] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.719777 9423 raft_consensus.cc:729] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d1ce8360324439966b7a8749dbdae4, State: Initialized, Role: FOLLOWER
I20260929 19:08:13.719933 9423 consensus_queue.cc:260] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:13.720206 9423 ts_tablet_manager.cc:1428] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260929 19:08:13.788731 9533 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:08:13.710864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:34375)
0929 19:08:13.711057 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.711100 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:13.711597 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:13.788503 (+ 76906us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":116,"threads_started":1}
W20260929 19:08:13.789018 9531 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:36761 (0 consecutive failures): Network error: Failed to ping master at 127.26.78.254:36761: Client connection negotiation failed: client connection to 127.26.78.254:36761: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:08:13.789359 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.789231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:48003)
0929 19:08:13.789288 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.789324 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 19:08:13.789752 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.789663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:59909)
0929 19:08:13.789712 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.789736 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 19:08:13.789922 9531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:08:13.816051 9535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:13.816267 9535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:13.816290 9535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:13.817422 9535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:13.817461 9535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:13.817487 9535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:08:13.819006 9535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:39335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=39763
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:13.819212 9535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:13.819387 9535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:13.824496 9542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.824541 9544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.824497 9541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.824728 9535 server_base.cc:1034] running on GCE node
I20260929 19:08:13.824950 9535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:13.825204 9535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:13.826375 9535 hybrid_clock.cc:648] HybridClock initialized: now 1790708893826359 us; error 46 us; skew 500 ppm
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:13.828917 9535 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:08:13.829365 9535 webserver.cc:460] Webserver started at http://127.26.78.194:39763/ using document root <none> and password file <none>
I20260929 19:08:13.829610 9535 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:13.829684 9535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:13.831095 9535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:13.831722 9551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.831892 9535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:08:13.831993 9535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "88d227551b3a4df4bcdd8426f8391311"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "1e188241ee4fdbb05243bf88ffcfcc07"
server_key_iv: "4027cbcca80ef50a01aed354417b642f"
server_key_version: "encryptionkey@0"
I20260929 19:08:13.832365 9535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:13.865072 9535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:13.867218 9535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:13.867398 9535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:13.867641 9535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:13.867961 9535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:13.868018 9535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.868063 9535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:13.868091 9535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.873930 9535 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39335
I20260929 19:08:13.873993 9664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39335 every 8 connection(s)
I20260929 19:08:13.874382 9535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:08:13.875458 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.875073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:38351)
0929 19:08:13.875230 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.875308 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
W20260929 19:08:13.875857 9665 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:36761 (0 consecutive failures): Network error: Failed to ping master at 127.26.78.254:36761: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
I20260929 19:08:13.876151 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.876044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:44289)
0929 19:08:13.876096 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.876120 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 19:08:13.876427 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:13.876341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:39489)
0929 19:08:13.876387 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:13.876409 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 19:08:13.876537 9665 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:08:13.884418 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9535
I20260929 19:08:13.884671 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:46595
--local_ip_for_outbound_sockets=127.26.78.195
--tserver_master_addrs=127.26.78.254:36761
--webserver_port=35149
--webserver_interface=127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:13.966146 9668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:13.966408 9668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:13.966452 9668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:13.967649 9668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:13.967723 9668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:13.967768 9668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:08:13.969252 9668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:46595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=35149
--tserver_master_addrs=127.26.78.254:36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:13.969548 9668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:13.969787 9668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:13.974794 9676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.974807 9674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:13.974805 9673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:13.975047 9668 server_base.cc:1034] running on GCE node
I20260929 19:08:13.975309 9668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:13.975562 9668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:13.976729 9668 hybrid_clock.cc:648] HybridClock initialized: now 1790708893976722 us; error 37 us; skew 500 ppm
Sep 29 19:08:13 dist-test-slave-xz51 krb5kdc[8233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:13.979204 9668 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:08:13.979696 9668 webserver.cc:460] Webserver started at http://127.26.78.195:35149/ using document root <none> and password file <none>
I20260929 19:08:13.979930 9668 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:13.980000 9668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:13.981339 9668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.982015 9683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:13.982192 9668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:13.982295 9668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "5cb074e7d3774637b9d35e016dcfd732"
format_stamp: "Formatted at 2026-09-29 19:08:12 on dist-test-slave-xz51"
server_key: "c154dd49fe64088f9383acf349d29b7c"
server_key_iv: "45701c30b0a5f8a320cd813463a6ccbd"
server_key_version: "encryptionkey@0"
I20260929 19:08:13.982718 9668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:14.001308 9668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:14.003508 9668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:14.003686 9668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:14.003947 9668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:14.004280 9668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:14.004336 9668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:14.004372 9668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:14.004405 9668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:14.010809 9668 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:46595
I20260929 19:08:14.010857 9796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:46595 every 8 connection(s)
I20260929 19:08:14.011226 9668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:08:14.012363 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:14.011876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:37161)
0929 19:08:14.012107 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:14.012216 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
W20260929 19:08:14.012791 9797 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:36761 (0 consecutive failures): Network error: Failed to ping master at 127.26.78.254:36761: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
I20260929 19:08:14.013111 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:14.013020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:39253)
0929 19:08:14.013068 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:14.013095 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 19:08:14.013381 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:14.013307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:37239)
0929 19:08:14.013345 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:14.013367 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 19:08:14.013502 9797 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:08:14.020160 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9668
I20260929 19:08:14.790557 9801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:14.790313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:49957)
0929 19:08:14.790446 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:14.790491 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260929 19:08:14.877104 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:14.876862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:57693)
0929 19:08:14.877007 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:14.877050 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":56,"threads_started":1}
I20260929 19:08:15.014148 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:15.013906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:49063)
0929 19:08:15.014050 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:15.014091 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":56,"threads_started":1}
I20260929 19:08:15.384318 9804 raft_consensus.cc:482] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:15.384446 9804 raft_consensus.cc:504] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:15.384752 9804 leader_election.cc:288] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335), 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
Sep 29 19:08:15 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.194@KRBTEST.COM
Sep 29 19:08:15 dist-test-slave-xz51 krb5kdc[8233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.195@KRBTEST.COM
I20260929 19:08:15.395651 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:15.384976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:46595 (local address 127.26.78.193:47669)
0929 19:08:15.385121 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:15.385132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:15.385186 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:15.386111 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:15.386112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:15.386467 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:15.386667 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:15.386674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.388301 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.388302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:15.389059 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:15.389061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.389690 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.389691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:15.389804 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:15.390485 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:15.390490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:15.392229 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:15.394145 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:15.394147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:15.394346 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:15.394606 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:15.394607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:15.394718 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:15.395112 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:15.395117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:15.395250 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 19:08:15.395360 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 19:08:15.395553 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":55,"thread_start_us":71,"threads_started":1}
I20260929 19:08:15.395663 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:15.385064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:47669 (local address 127.26.78.195:46595)
0929 19:08:15.385282 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:15.385290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:15.385342 (+ 52us) server_negotiation.cc:407] Connection header received
0929 19:08:15.385898 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:15.385900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:15.385981 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:15.386125 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:15.386771 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.388158 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.389168 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.389583 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.389716 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:15.392336 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:15.392342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:15.392343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:15.392362 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:15.394050 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394437 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:15.394442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:15.394443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:15.394530 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394787 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:15.394788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:15.394789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:15.395029 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:15.395119 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:15.395421 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 19:08:15.395544 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":47,"threads_started":1}
I20260929 19:08:15.395720 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:15.385086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:55087 (local address 127.26.78.194:39335)
0929 19:08:15.385248 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:15.385269 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:15.385290 (+ 21us) server_negotiation.cc:407] Connection header received
0929 19:08:15.385744 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:15.385746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:15.385789 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:15.385957 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:15.386806 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.388159 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.389171 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.389589 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.389731 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:15.392000 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:15.392005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:15.392005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:15.392018 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:15.393968 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394425 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:15.394429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:15.394429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:15.394524 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394792 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:15.394793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:15.394794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:15.395029 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:15.395119 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:15.395436 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 19:08:15.395591 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":56,"threads_started":1}
I20260929 19:08:15.396425 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:15.384946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39335 (local address 127.26.78.193:55087)
0929 19:08:15.385105 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:15.385117 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:15.385180 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:15.386024 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:15.386027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:15.386454 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:15.386666 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:15.386673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.388254 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.388255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:15.389060 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:15.389062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:15.389693 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:15.389694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:15.389805 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:15.390415 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:15.390423 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:15.391859 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:15.394066 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:15.394075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:15.394337 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:15.394595 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:15.394597 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:15.394597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:15.394717 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:15.395110 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:15.395117 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:15.395250 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 19:08:15.395370 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 19:08:15.395634 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":136,"thread_start_us":82,"threads_started":1}
W20260929 19:08:15.396503 9420 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
W20260929 19:08:15.397104 9417 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:15.397166 9417 leader_election.cc:302] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63d1ce8360324439966b7a8749dbdae4; no voters: 5cb074e7d3774637b9d35e016dcfd732, 88d227551b3a4df4bcdd8426f8391311
I20260929 19:08:15.397286 9804 raft_consensus.cc:2726] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:08:15.791188 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:15.791032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:54697)
0929 19:08:15.791104 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:15.791146 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 19:08:15.877838 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:15.877577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:58875)
0929 19:08:15.877728 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:15.877774 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260929 19:08:16.014755 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:16.014558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:52157)
0929 19:08:16.014683 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:16.014717 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260929 19:08:16.792057 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:16.791778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:54889)
0929 19:08:16.791933 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:16.791997 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260929 19:08:16.878480 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:16.878292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:45919)
0929 19:08:16.878404 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:16.878439 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260929 19:08:17.015384 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:17.015180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:44369)
0929 19:08:17.015307 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:17.015340 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260929 19:08:17.153405 9814 raft_consensus.cc:482] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:17.153501 9814 raft_consensus.cc:504] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:17.153688 9814 leader_election.cc:288] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335), 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
W20260929 19:08:17.154156 9420 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
W20260929 19:08:17.154247 9417 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:17.154326 9417 leader_election.cc:302] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63d1ce8360324439966b7a8749dbdae4; no voters: 5cb074e7d3774637b9d35e016dcfd732, 88d227551b3a4df4bcdd8426f8391311
I20260929 19:08:17.154426 9814 raft_consensus.cc:2726] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:08:17.792750 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:17.792540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:46193)
0929 19:08:17.792671 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:17.792702 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260929 19:08:17.879180 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:17.878962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:47599)
0929 19:08:17.879097 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:17.879135 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260929 19:08:18.016054 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:18.015841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:56307)
0929 19:08:18.015981 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:18.016014 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260929 19:08:18.793583 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:18.793286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:47721)
0929 19:08:18.793479 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:18.793537 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
I20260929 19:08:18.879869 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:18.879665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:48207)
0929 19:08:18.879787 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:18.879824 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
I20260929 19:08:19.016734 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:19.016545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:60199)
0929 19:08:19.016658 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:19.016692 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260929 19:08:19.689678 9821 raft_consensus.cc:482] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:19.689771 9821 raft_consensus.cc:504] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:19.689989 9821 leader_election.cc:288] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335), 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
W20260929 19:08:19.690374 9420 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
W20260929 19:08:19.690457 9417 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:19.690538 9417 leader_election.cc:302] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63d1ce8360324439966b7a8749dbdae4; no voters: 5cb074e7d3774637b9d35e016dcfd732, 88d227551b3a4df4bcdd8426f8391311
I20260929 19:08:19.690600 9821 raft_consensus.cc:2726] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:08:19.794348 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:19.794125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:58705)
0929 19:08:19.794251 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:19.794294 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260929 19:08:19.880496 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:19.880281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:36983)
0929 19:08:19.880411 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:19.880445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260929 19:08:20.017371 9824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:20.017181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:45603)
0929 19:08:20.017294 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:20.017325 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260929 19:08:20.795040 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:20.794813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:36631)
0929 19:08:20.794949 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:20.794994 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260929 19:08:20.881193 9826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:20.880996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:52031)
0929 19:08:20.881119 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:20.881150 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260929 19:08:21.018224 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:21.017950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:54603)
0929 19:08:21.018105 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:21.018161 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260929 19:08:21.795819 9828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:21.795564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:44733)
0929 19:08:21.795728 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:21.795772 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260929 19:08:21.881932 9829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:21.881703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:36075)
0929 19:08:21.881811 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:21.881887 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260929 19:08:22.018996 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:22.018715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:55673)
0929 19:08:22.018890 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:22.018945 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260929 19:08:22.796566 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:22.796336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:36675)
0929 19:08:22.796474 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:22.796518 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260929 19:08:22.882658 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:22.882445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:41099)
0929 19:08:22.882572 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:22.882612 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260929 19:08:23.019678 9833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:23.019470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:42621)
0929 19:08:23.019597 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:23.019636 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260929 19:08:23.685521 9834 raft_consensus.cc:482] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:23.685607 9834 raft_consensus.cc:504] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:23.685771 9834 leader_election.cc:288] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335), 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
W20260929 19:08:23.686137 9417 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
W20260929 19:08:23.686204 9420 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:23.686231 9417 leader_election.cc:302] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63d1ce8360324439966b7a8749dbdae4; no voters: 5cb074e7d3774637b9d35e016dcfd732, 88d227551b3a4df4bcdd8426f8391311
I20260929 19:08:23.686298 9834 raft_consensus.cc:2726] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:08:23.797326 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:23.797122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:38713)
0929 19:08:23.797244 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:23.797280 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260929 19:08:23.883401 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:23.883160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:42863)
0929 19:08:23.883298 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:23.883356 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260929 19:08:24.020503 9837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:24.020225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:54225)
0929 19:08:24.020396 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:24.020437 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260929 19:08:24.798079 9838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:24.797861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:46467)
0929 19:08:24.797995 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:24.798032 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260929 19:08:24.884104 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:24.883904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:48029)
0929 19:08:24.884023 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:24.884060 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260929 19:08:25.021202 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:25.021002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:54461)
0929 19:08:25.021121 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:25.021159 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260929 19:08:25.798795 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:25.798561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:42727)
0929 19:08:25.798709 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:25.798748 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260929 19:08:25.884779 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:25.884568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:45495)
0929 19:08:25.884696 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:25.884734 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260929 19:08:26.021992 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:26.021678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:60201)
0929 19:08:26.021893 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:26.021943 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260929 19:08:26.244057 9844 raft_consensus.cc:482] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:26.244150 9844 raft_consensus.cc:504] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88d227551b3a4df4bcdd8426f8391311" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39335 } } peers { permanent_uuid: "5cb074e7d3774637b9d35e016dcfd732" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 46595 } } peers { permanent_uuid: "63d1ce8360324439966b7a8749dbdae4" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 34779 } }
I20260929 19:08:26.244359 9844 leader_election.cc:288] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335), 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595)
W20260929 19:08:26.244810 9417 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88d227551b3a4df4bcdd8426f8391311 (127.26.78.194:39335): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
W20260929 19:08:26.244895 9420 leader_election.cc:341] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cb074e7d3774637b9d35e016dcfd732 (127.26.78.195:46595): Not found: Tablet not found: 2b0c307e91174864a3e9fdaa884c570c
I20260929 19:08:26.244961 9420 leader_election.cc:302] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63d1ce8360324439966b7a8749dbdae4; no voters: 5cb074e7d3774637b9d35e016dcfd732, 88d227551b3a4df4bcdd8426f8391311
I20260929 19:08:26.245056 9844 raft_consensus.cc:2726] T 2b0c307e91174864a3e9fdaa884c570c P 63d1ce8360324439966b7a8749dbdae4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:08:26.799485 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:26.799271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:40797)
0929 19:08:26.799400 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:26.799438 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260929 19:08:26.885465 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:26.885264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:42139)
0929 19:08:26.885379 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:26.885420 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260929 19:08:27.022735 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:27.022503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:44513)
0929 19:08:27.022644 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:27.022688 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":58,"threads_started":1}
I20260929 19:08:27.800248 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:27.799988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:52395)
0929 19:08:27.800145 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:27.800201 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
I20260929 19:08:27.886211 9849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:27.886006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:34719)
0929 19:08:27.886122 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:27.886165 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260929 19:08:28.023559 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:28.023285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:57757)
0929 19:08:28.023467 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:28.023511 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260929 19:08:28.801060 9851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:28.800862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.193:57625)
0929 19:08:28.800982 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:28.801019 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260929 19:08:28.886998 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:28.886757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.194:58895)
0929 19:08:28.886903 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:28.886951 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260929 19:08:29.024564 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:29.024162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:36761 (local address 127.26.78.195:37419)
0929 19:08:29.024474 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:29.024518 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260929 19:08:29.024648 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 9372
I20260929 19:08:29.029495 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 9535
I20260929 19:08:29.034556 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 9668
2026-09-29T19:08:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18205 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:08:29 dist-test-slave-xz51 krb5kdc[9856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:08:29 dist-test-slave-xz51 krb5kdc[9856](info): set up 2 sockets
Sep 29 19:08:29 dist-test-slave-xz51 krb5kdc[9856](info): commencing operation
krb5kdc: starting...
W20260929 19:08:30.074039 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:08:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:08:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:08:30.195180 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40043
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:40043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:30.275576 9869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:30.275794 9869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:30.275816 9869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:30.277076 9869 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 19:08:30.277117 9869 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 19:08:30.277129 9869 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 19:08:30.277141 9869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:30.277194 9869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:30.277220 9869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:30.277245 9869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:30.277276 9869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:30.278837 9869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:40043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:30.279088 9869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:30.279335 9869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:30.284821 9877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:30.284921 9874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:30.284989 9869 server_base.cc:1034] running on GCE node
W20260929 19:08:30.284945 9875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:30.285383 9869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:30.285642 9869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:30.286774 9869 hybrid_clock.cc:648] HybridClock initialized: now 1790708910286762 us; error 29 us; skew 500 ppm
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:30.289111 9869 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:30.289505 9869 webserver.cc:460] Webserver started at http://127.26.78.254:34437/ using document root <none> and password file <none>
I20260929 19:08:30.289700 9869 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:30.289739 9869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:30.289899 9869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:30.290904 9869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "79485f4709d94ab89f1dc0d99740cd37"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "72cbdd5f80d6b12207b551bbc1110613"
server_key_iv: "8601863260fa2e34d7a0af8842f1b208"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.291198 9869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "79485f4709d94ab89f1dc0d99740cd37"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "72cbdd5f80d6b12207b551bbc1110613"
server_key_iv: "8601863260fa2e34d7a0af8842f1b208"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.292481 9869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:08:30.293152 9884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.293299 9869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:08:30.293396 9869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "79485f4709d94ab89f1dc0d99740cd37"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "72cbdd5f80d6b12207b551bbc1110613"
server_key_iv: "8601863260fa2e34d7a0af8842f1b208"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.293453 9869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:30.318252 9869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:30.320685 9869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:30.320806 9869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:30.324692 9869 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:40043
I20260929 19:08:30.324776 9936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:40043 every 8 connection(s)
I20260929 19:08:30.325119 9869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:30.326093 9937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:30.328390 9937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Bootstrap starting.
I20260929 19:08:30.328975 9937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:30.329265 9937 log.cc:824] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:30.330057 9937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: No bootstrap required, opened a new log
I20260929 19:08:30.330070 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9869
I20260929 19:08:30.330150 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:08:30.330377 26939 external_mini_cluster.cc:1428] Setting key 58e1f775aafc9b082d9f7b91eb3b2c39
I20260929 19:08:30.332019 9937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:30.332193 9937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:30.332234 9937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79485f4709d94ab89f1dc0d99740cd37, State: Initialized, Role: FOLLOWER
I20260929 19:08:30.332410 9937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:30.332513 9937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:30.332551 9937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:30.332603 9937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:30.333256 9937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:30.333344 9937 leader_election.cc:302] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79485f4709d94ab89f1dc0d99740cd37; no voters:
I20260929 19:08:30.333527 9937 leader_election.cc:288] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:08:30.333556 9942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:30.333664 9942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 LEADER]: Becoming Leader. State: Replica: 79485f4709d94ab89f1dc0d99740cd37, State: Running, Role: LEADER
I20260929 19:08:30.333797 9937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:30.333797 9942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:30.334250 9943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79485f4709d94ab89f1dc0d99740cd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } } }
I20260929 19:08:30.334369 9943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:30.334262 9944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79485f4709d94ab89f1dc0d99740cd37. Latest consensus state: current_term: 1 leader_uuid: "79485f4709d94ab89f1dc0d99740cd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } } }
I20260929 19:08:30.334506 9944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:30.334810 9949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:30.335330 9949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:30.337159 9949 catalog_manager.cc:1347] Generated new cluster ID: 8db1d950e8674e519f70f5d0ff6c3815
I20260929 19:08:30.337216 9949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:30.342213 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.330900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.26.78.254:40043)
0929 19:08:30.331067 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.331075 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.331101 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:08:30.331999 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.332006 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.332051 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.332169 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:30.332968 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.334604 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.335713 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.336314 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.336428 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.338971 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:30.338975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:30.338976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:30.338988 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:30.340658 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.341040 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.341043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.341043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.341154 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.341359 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.341359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.341360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.341610 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:30.341786 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.341898 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.342049 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260929 19:08:30.354264 9949 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:08:30.355026 9949 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:30.362186 9949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Generated new TSK 0
I20260929 19:08:30.362390 9949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:08:30.392864 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40043
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:30.492470 9963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:30.492714 9963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:30.492754 9963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:30.494016 9963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:30.494076 9963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:30.494120 9963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:08:30.495625 9963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:30.495965 9963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:30.496202 9963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:30.501479 9968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:30.501474 9969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:30.501542 9963 server_base.cc:1034] running on GCE node
W20260929 19:08:30.501482 9971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:30.501930 9963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:30.502194 9963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:30.503440 9963 hybrid_clock.cc:648] HybridClock initialized: now 1790708910503419 us; error 34 us; skew 500 ppm
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:30.506038 9963 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:08:30.506455 9963 webserver.cc:460] Webserver started at http://127.26.78.193:40137/ using document root <none> and password file <none>
I20260929 19:08:30.506753 9963 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:30.506826 9963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:30.506975 9963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:30.508010 9963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "51033524a81e41319793b7430f64cb5e"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "5fa14328172921aa79019ca436f6a6ea"
server_key_iv: "fe50f906d6aa12f9c57de2c9cb7478d5"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.508369 9963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "51033524a81e41319793b7430f64cb5e"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "5fa14328172921aa79019ca436f6a6ea"
server_key_iv: "fe50f906d6aa12f9c57de2c9cb7478d5"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.509682 9963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:08:30.510456 9978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.510618 9963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:08:30.510713 9963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "51033524a81e41319793b7430f64cb5e"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "5fa14328172921aa79019ca436f6a6ea"
server_key_iv: "fe50f906d6aa12f9c57de2c9cb7478d5"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.510790 9963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:30.526691 9963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:30.528813 9963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:30.528982 9963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:30.529209 9963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:30.529507 9963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:30.529556 9963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.529599 9963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:30.529624 9963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.535322 9963 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:36697
I20260929 19:08:30.535413 10091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:36697 every 8 connection(s)
I20260929 19:08:30.535712 9963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:08:30.538508 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 9963
I20260929 19:08:30.538594 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:08:30.538794 26939 external_mini_cluster.cc:1428] Setting key 758b69023d030b80532bb68e1cdc8cc0
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:30.546248 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.536426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:56065 (local address 127.26.78.254:40043)
0929 19:08:30.536469 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.536471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.536683 (+ 212us) server_negotiation.cc:407] Connection header received
0929 19:08:30.537243 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.537244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.537259 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.537319 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:30.538344 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.538997 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.540501 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.540723 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.540790 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.543306 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:30.543307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:30.543308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:30.543315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:30.544757 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.545262 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.545263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.545264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.545312 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.545614 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.545615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.545615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.545712 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:30.545758 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.546115 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.546185 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:08:30.546568 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.536393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:56065)
0929 19:08:30.536571 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:30.536606 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:30.537138 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:30.537388 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:30.537392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:30.538043 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:30.538238 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.538243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.539100 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.539101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.540407 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.540410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.540814 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.540815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.540950 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.541479 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:30.541487 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:30.543206 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:30.544847 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:30.544854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:30.544857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:30.545189 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:30.545391 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:30.545392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:30.545392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:30.545542 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:30.545769 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:30.545775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:30.545973 (+ 198us) client_negotiation.cc:769] Sending connection context
0929 19:08:30.546101 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 19:08:30.546279 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260929 19:08:30.547133 10092 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40043
I20260929 19:08:30.547225 10092 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:30.547400 10092 heartbeater.cc:511] Master 127.26.78.254:40043 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
I20260929 19:08:30.547907 9901 ts_manager.cc:194] Registered new tserver with Master: 51033524a81e41319793b7430f64cb5e (127.26.78.193:36697)
I20260929 19:08:30.548489 9901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:56065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:08:30.565115 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40043
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:30.647049 10097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:30.647276 10097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:30.647316 10097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:30.648468 10097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:30.648532 10097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:30.648578 10097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:08:30.650066 10097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:30.650317 10097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:30.650543 10097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:30.655465 10105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:30.655476 10102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:30.655484 10103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:30.655683 10097 server_base.cc:1034] running on GCE node
I20260929 19:08:30.655915 10097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:30.656152 10097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:30.657315 10097 hybrid_clock.cc:648] HybridClock initialized: now 1790708910657307 us; error 37 us; skew 500 ppm
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:30.659725 10097 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:08:30.660137 10097 webserver.cc:460] Webserver started at http://127.26.78.194:39337/ using document root <none> and password file <none>
I20260929 19:08:30.660355 10097 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:30.660418 10097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:30.660560 10097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:30.661602 10097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "d23632b2c390420dbf7d4e8489c35a55"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "8a0df79d4def1fb89eee0c677601ca60"
server_key_iv: "ce59c0e7754a39e22981c5f0c24763af"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.662099 10097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "d23632b2c390420dbf7d4e8489c35a55"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "8a0df79d4def1fb89eee0c677601ca60"
server_key_iv: "ce59c0e7754a39e22981c5f0c24763af"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.663558 10097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:30.664268 10112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.664434 10097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:08:30.664534 10097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d23632b2c390420dbf7d4e8489c35a55"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "8a0df79d4def1fb89eee0c677601ca60"
server_key_iv: "ce59c0e7754a39e22981c5f0c24763af"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.664605 10097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:30.677325 10097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:30.679764 10097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:30.679944 10097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:30.680197 10097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:30.680517 10097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:30.680574 10097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.680619 10097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:30.680727 10097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.686977 10097 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:38383
I20260929 19:08:30.687013 10225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:38383 every 8 connection(s)
I20260929 19:08:30.687388 10097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:08:30.689687 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10097
I20260929 19:08:30.689765 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:08:30.689975 26939 external_mini_cluster.cc:1428] Setting key a027ddb767c53592b4c4264d5c2be04a
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:30.698501 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.688144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:32799 (local address 127.26.78.254:40043)
0929 19:08:30.688217 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.688223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.688428 (+ 205us) server_negotiation.cc:407] Connection header received
0929 19:08:30.688942 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.688943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.688953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.689012 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:30.690202 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.690903 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.692309 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.692502 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.692560 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.695134 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:30.695135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:30.695136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:30.695143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:30.696961 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.697380 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.697382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.697382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.697453 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.697744 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.697746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.697746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.697940 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:30.698028 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.698284 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.698381 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:08:30.698513 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.688113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:32799)
0929 19:08:30.688291 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:30.688335 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:30.688856 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:30.689070 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:30.689073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:30.689708 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:30.690071 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.690078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.691007 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.691008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.692212 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.692216 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.692589 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.692590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.692762 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.693310 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:30.693322 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:30.695036 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:30.697046 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:30.697053 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:30.697067 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:30.697298 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:30.697534 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:30.697535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:30.697536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:30.697666 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:30.698013 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:30.698018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:30.698135 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 19:08:30.698225 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:08:30.698349 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260929 19:08:30.699057 10226 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40043
I20260929 19:08:30.699149 10226 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:30.699365 10226 heartbeater.cc:511] Master 127.26.78.254:40043 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
I20260929 19:08:30.699790 9901 ts_manager.cc:194] Registered new tserver with Master: d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
I20260929 19:08:30.700162 9901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:32799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:08:30.716755 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40043
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:30.798702 10231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:30.798919 10231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:30.798945 10231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:30.800262 10231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:30.800314 10231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:30.800364 10231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:08:30.801868 10231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:30.802119 10231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:30.802364 10231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:30.807647 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:30.807647 10237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:30.807675 10239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:30.807787 10231 server_base.cc:1034] running on GCE node
I20260929 19:08:30.807987 10231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:30.808252 10231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:30.809420 10231 hybrid_clock.cc:648] HybridClock initialized: now 1790708910809397 us; error 36 us; skew 500 ppm
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:30.811971 10231 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:08:30.812438 10231 webserver.cc:460] Webserver started at http://127.26.78.195:43063/ using document root <none> and password file <none>
I20260929 19:08:30.812698 10231 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:30.812776 10231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:30.812925 10231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:08:30.814117 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "3f22ed8381304f80bdd36fcb7079be94"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "1159d03b4f0c4f84aa70ff1e8eecb8a8"
server_key_iv: "ee9c9d24a5f94ebde7a6c8e31cbb8f8a"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.814473 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "3f22ed8381304f80bdd36fcb7079be94"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "1159d03b4f0c4f84aa70ff1e8eecb8a8"
server_key_iv: "ee9c9d24a5f94ebde7a6c8e31cbb8f8a"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.815809 10231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:08:30.816502 10246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.816663 10231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:08:30.816798 10231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "3f22ed8381304f80bdd36fcb7079be94"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "1159d03b4f0c4f84aa70ff1e8eecb8a8"
server_key_iv: "ee9c9d24a5f94ebde7a6c8e31cbb8f8a"
server_key_version: "encryptionkey@0"
I20260929 19:08:30.816872 10231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:30.842237 10231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:30.844656 10231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:30.844825 10231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:30.845073 10231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:30.845402 10231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:30.845453 10231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.845489 10231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:30.845515 10231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:30.851238 10231 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:45581
I20260929 19:08:30.851285 10359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:45581 every 8 connection(s)
I20260929 19:08:30.851612 10231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:08:30.851774 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10231
I20260929 19:08:30.851879 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:08:30.852075 26939 external_mini_cluster.cc:1428] Setting key 3b73fa11652665ae805ad534a4c69282
Sep 29 19:08:30 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:30.860777 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.852468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:49675 (local address 127.26.78.254:40043)
0929 19:08:30.852515 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.852518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.852726 (+ 208us) server_negotiation.cc:407] Connection header received
0929 19:08:30.853248 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.853249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.853261 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.853319 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:30.854331 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.854837 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.855840 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.856029 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.856121 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.858198 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:30.858199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:30.858200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:30.858207 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:30.859401 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.859824 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.859825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.859826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.859884 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.860124 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.860125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.860125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.860238 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:30.860304 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.860606 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.860686 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:08:30.860985 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.852409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:49675)
0929 19:08:30.852592 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:30.852634 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:30.853143 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:30.853390 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:30.853394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:30.854039 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:30.854234 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.854240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.854930 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.854932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.855731 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.855733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.856130 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.856131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.856252 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.856701 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:30.856709 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:30.858105 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:30.859512 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:30.859519 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:30.859521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:30.859738 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:30.859949 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:30.859950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:30.859951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:30.860052 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:30.860308 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:30.860314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:30.860481 (+ 167us) client_negotiation.cc:769] Sending connection context
0929 19:08:30.860584 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 19:08:30.860739 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260929 19:08:30.861459 10360 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40043
I20260929 19:08:30.861538 10360 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:30.861680 10360 heartbeater.cc:511] Master 127.26.78.254:40043 requested a full tablet report, sending...
I20260929 19:08:30.862042 9901 ts_manager.cc:194] Registered new tserver with Master: 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581)
I20260929 19:08:30.862491 9901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:49675
I20260929 19:08:30.863649 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:08:30.870939 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.864902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.26.78.254:40043)
0929 19:08:30.864945 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.864947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.865039 (+ 92us) server_negotiation.cc:407] Connection header received
0929 19:08:30.865086 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.865087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.865094 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.865140 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:30.865639 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.866172 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.866906 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.867085 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.867148 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.868563 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:30.868564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:30.868564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:30.868570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:30.870015 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.870305 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.870306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.870306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.870353 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:30.870496 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:30.870497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:30.870497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:30.870616 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:30.870684 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.870773 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.870828 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:08:30.872500 9901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:08:30.874205 9901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:08:30.880438 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.876481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.26.78.194:38383)
0929 19:08:30.876805 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.876809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.876820 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:08:30.876899 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.876901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.876919 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.876997 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:30.877192 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.878039 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.879364 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.880005 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.880099 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.880159 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.880319 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.880358 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":84,"threads_started":1}
I20260929 19:08:30.880437 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.876368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38383 (local address 127.0.0.1:46050)
0929 19:08:30.876691 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:30.876732 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:30.876799 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:30.877045 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:30.877047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:30.877049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:30.877113 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.877115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.878138 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.878139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.879266 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.879268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.880104 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.880105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.880193 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.880207 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:08:30.880300 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 19:08:30.880354 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":4,"thread_start_us":83,"threads_started":1}
I20260929 19:08:30.880812 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.876369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:45581 (local address 127.0.0.1:46056)
0929 19:08:30.876690 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:30.876734 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:30.876801 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:30.877218 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:30.877219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:30.877221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:30.877279 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.877281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.878614 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.878615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.879700 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.879702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.880607 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.880608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.880664 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.880667 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:08:30.880725 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:08:30.880760 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":29,"thread_start_us":57,"threads_started":1}
I20260929 19:08:30.880893 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.876684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.26.78.193:36697)
0929 19:08:30.876878 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.876883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.876982 (+ 99us) server_negotiation.cc:407] Connection header received
0929 19:08:30.877026 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.877028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.877044 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.877113 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:30.877278 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.878506 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.879590 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.880274 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.880344 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.880397 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.880740 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.880786 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260929 19:08:30.880899 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.876464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.26.78.195:45581)
0929 19:08:30.876712 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:30.876717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:30.876899 (+ 182us) server_negotiation.cc:407] Connection header received
0929 19:08:30.876940 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:30.876942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:30.876957 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:30.877025 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:30.877360 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.878504 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.879779 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.880463 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.880525 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.880575 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:30.880755 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 19:08:30.880802 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":98,"threads_started":1}
I20260929 19:08:30.880668 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:30.876568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36697 (local address 127.0.0.1:54254)
0929 19:08:30.876914 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:30.876923 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:30.876971 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:30.877123 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:30.877124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:30.877126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:30.877162 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.877163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.878625 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.878626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.879500 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:30.879502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:30.880434 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:30.880435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:30.880500 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:30.880505 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:30.880551 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:08:30.880594 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":62,"threads_started":1}
I20260929 19:08:30.881767 10160 tablet_service.cc:1467] Processing CreateTablet for tablet 2f6d47976fb244a2856edf7b162ffa06 (DEFAULT_TABLE table=test-table [id=bc661177ff4c4a46baafa03f473fb1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:30.882182 10160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f6d47976fb244a2856edf7b162ffa06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:30.882279 10294 tablet_service.cc:1467] Processing CreateTablet for tablet 2f6d47976fb244a2856edf7b162ffa06 (DEFAULT_TABLE table=test-table [id=bc661177ff4c4a46baafa03f473fb1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:30.882342 10026 tablet_service.cc:1467] Processing CreateTablet for tablet 2f6d47976fb244a2856edf7b162ffa06 (DEFAULT_TABLE table=test-table [id=bc661177ff4c4a46baafa03f473fb1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:30.882637 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f6d47976fb244a2856edf7b162ffa06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:30.882637 10026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f6d47976fb244a2856edf7b162ffa06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:30.884904 10377 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Bootstrap starting.
I20260929 19:08:30.885527 10377 tablet_bootstrap.cc:654] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:30.886117 10378 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Bootstrap starting.
I20260929 19:08:30.886150 10379 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Bootstrap starting.
I20260929 19:08:30.886198 10377 log.cc:824] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:30.886801 10379 tablet_bootstrap.cc:654] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:30.886830 10378 tablet_bootstrap.cc:654] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:30.887034 10377 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: No bootstrap required, opened a new log
I20260929 19:08:30.887116 10377 ts_tablet_manager.cc:1397] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:08:30.887363 10379 log.cc:824] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:30.887372 10378 log.cc:824] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:30.888070 10379 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: No bootstrap required, opened a new log
I20260929 19:08:30.888070 10378 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: No bootstrap required, opened a new log
I20260929 19:08:30.888139 10378 ts_tablet_manager.cc:1397] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:08:30.888288 10379 ts_tablet_manager.cc:1397] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:08:30.888855 10377 raft_consensus.cc:348] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:30.889014 10377 raft_consensus.cc:374] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:30.889057 10377 raft_consensus.cc:729] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23632b2c390420dbf7d4e8489c35a55, State: Initialized, Role: FOLLOWER
I20260929 19:08:30.889161 10377 consensus_queue.cc:260] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:30.889434 10377 ts_tablet_manager.cc:1428] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:08:30.889576 10226 heartbeater.cc:503] Master 127.26.78.254:40043 was elected leader, sending a full tablet report...
I20260929 19:08:30.890133 10378 raft_consensus.cc:348] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:30.890321 10378 raft_consensus.cc:374] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:30.890354 10378 raft_consensus.cc:729] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f22ed8381304f80bdd36fcb7079be94, State: Initialized, Role: FOLLOWER
I20260929 19:08:30.890473 10378 consensus_queue.cc:260] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:30.890578 10379 raft_consensus.cc:348] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:30.890759 10378 ts_tablet_manager.cc:1428] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:08:30.890754 10379 raft_consensus.cc:374] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:30.890797 10379 raft_consensus.cc:729] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51033524a81e41319793b7430f64cb5e, State: Initialized, Role: FOLLOWER
I20260929 19:08:30.890808 10360 heartbeater.cc:503] Master 127.26.78.254:40043 was elected leader, sending a full tablet report...
I20260929 19:08:30.890914 10379 consensus_queue.cc:260] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:30.891165 10379 ts_tablet_manager.cc:1428] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:08:30.891309 10092 heartbeater.cc:503] Master 127.26.78.254:40043 was elected leader, sending a full tablet report...
W20260929 19:08:30.937952 10227 tablet.cc:2369] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:08:31.036443 10093 tablet.cc:2369] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:08:31.102406 10361 tablet.cc:2369] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:08:31.218026 10385 raft_consensus.cc:482] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:31.218155 10385 raft_consensus.cc:504] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:31.218578 10385 leader_election.cc:288] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581), d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
I20260929 19:08:31.221624 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:31.218882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:45293 (local address 127.26.78.194:38383)
0929 19:08:31.218928 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:31.218931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:31.219028 (+ 97us) server_negotiation.cc:407] Connection header received
0929 19:08:31.219107 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:31.219108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:31.219116 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:31.219186 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:31.219358 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.219845 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.220916 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.221276 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.221366 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:31.221374 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:31.221522 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 19:08:31.221558 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:08:31.221637 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:31.218789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38383 (local address 127.26.78.193:45293)
0929 19:08:31.218942 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:31.218953 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:31.219038 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:31.219193 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:31.219193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:31.219196 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:31.219288 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:31.219290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.219944 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.219945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:31.220821 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:31.220824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.221366 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.221367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:31.221436 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:31.221441 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:31.221504 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:08:31.221546 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260929 19:08:31.221725 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:31.218921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:49389 (local address 127.26.78.195:45581)
0929 19:08:31.218964 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:31.218966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:31.219029 (+ 63us) server_negotiation.cc:407] Connection header received
0929 19:08:31.219100 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:31.219100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:31.219109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:31.219174 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:31.219339 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.219935 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.220976 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.221369 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.221454 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:31.221463 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:31.221618 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 19:08:31.221654 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:08:31.221753 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:31.218805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:45581 (local address 127.26.78.193:49389)
0929 19:08:31.218948 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:31.218959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:31.219031 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:31.219179 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:31.219180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:31.219182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:31.219260 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:31.219262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.220026 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.220027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:31.220899 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:31.220901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.221454 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.221455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:31.221527 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:31.221531 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:08:31.221597 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:08:31.221638 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260929 19:08:31.222127 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6d47976fb244a2856edf7b162ffa06" candidate_uuid: "51033524a81e41319793b7430f64cb5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f22ed8381304f80bdd36fcb7079be94" is_pre_election: true
I20260929 19:08:31.222163 10180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6d47976fb244a2856edf7b162ffa06" candidate_uuid: "51033524a81e41319793b7430f64cb5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d23632b2c390420dbf7d4e8489c35a55" is_pre_election: true
I20260929 19:08:31.222299 10314 raft_consensus.cc:2454] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51033524a81e41319793b7430f64cb5e in term 0.
I20260929 19:08:31.222299 10180 raft_consensus.cc:2454] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51033524a81e41319793b7430f64cb5e in term 0.
I20260929 19:08:31.222572 9982 leader_election.cc:302] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51033524a81e41319793b7430f64cb5e, d23632b2c390420dbf7d4e8489c35a55; no voters:
I20260929 19:08:31.222754 10385 raft_consensus.cc:2781] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:08:31.222817 10385 raft_consensus.cc:482] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:08:31.222836 10385 raft_consensus.cc:3037] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:31.223531 10385 raft_consensus.cc:504] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:31.223676 10385 leader_election.cc:288] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 1 election: Requested vote from peers 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581), d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
I20260929 19:08:31.223824 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6d47976fb244a2856edf7b162ffa06" candidate_uuid: "51033524a81e41319793b7430f64cb5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f22ed8381304f80bdd36fcb7079be94"
I20260929 19:08:31.223879 10180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6d47976fb244a2856edf7b162ffa06" candidate_uuid: "51033524a81e41319793b7430f64cb5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d23632b2c390420dbf7d4e8489c35a55"
I20260929 19:08:31.223913 10314 raft_consensus.cc:3037] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:31.223960 10180 raft_consensus.cc:3037] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:31.224809 10314 raft_consensus.cc:2454] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51033524a81e41319793b7430f64cb5e in term 1.
I20260929 19:08:31.224810 10180 raft_consensus.cc:2454] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51033524a81e41319793b7430f64cb5e in term 1.
I20260929 19:08:31.224998 9982 leader_election.cc:302] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51033524a81e41319793b7430f64cb5e, d23632b2c390420dbf7d4e8489c35a55; no voters:
I20260929 19:08:31.225116 10385 raft_consensus.cc:2781] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:31.225276 10385 raft_consensus.cc:686] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 1 LEADER]: Becoming Leader. State: Replica: 51033524a81e41319793b7430f64cb5e, State: Running, Role: LEADER
I20260929 19:08:31.225380 10385 consensus_queue.cc:237] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:31.226269 9901 catalog_manager.cc:5495] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 51033524a81e41319793b7430f64cb5e (127.26.78.193). New cstate: current_term: 1 leader_uuid: "51033524a81e41319793b7430f64cb5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } health_report { overall_health: HEALTHY } } }
I20260929 19:08:31.233292 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:31.230904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.26.78.193:36697)
0929 19:08:31.230946 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:31.230948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:31.230956 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:08:31.231063 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:31.231064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:31.231082 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:31.231161 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:08:31.231293 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.231768 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.232534 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:31.232735 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:31.232804 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:31.232975 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:08:31.233082 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:08:31.233173 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:31.233193 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 19:08:31.233212 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:08:31.234979 10180 raft_consensus.cc:1261] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Refusing update from remote peer 51033524a81e41319793b7430f64cb5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:31.235005 10314 raft_consensus.cc:1261] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Refusing update from remote peer 51033524a81e41319793b7430f64cb5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:31.235291 10388 consensus_queue.cc:1035] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:31.235417 10385 consensus_queue.cc:1035] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:31.237596 10392 mvcc.cc:205] Tried to move back new op lower bound from 7334743700416126976 to 7334743700379783168. Current Snapshot: MvccSnapshot[applied={T|T < 7334743700416126976}]
I20260929 19:08:31.237874 10393 mvcc.cc:205] Tried to move back new op lower bound from 7334743700416126976 to 7334743700379783168. Current Snapshot: MvccSnapshot[applied={T|T < 7334743700416126976}]
I20260929 19:08:31.238248 10391 mvcc.cc:205] Tried to move back new op lower bound from 7334743700416126976 to 7334743700379783168. Current Snapshot: MvccSnapshot[applied={T|T < 7334743700416126976 or (T in {7334743700416126976})}]
I20260929 19:08:31.240058 9901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:31.240155 9901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:31.240881 9901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Sending DeleteTablet for 3 replicas of tablet 2f6d47976fb244a2856edf7b162ffa06
I20260929 19:08:31.241221 10160 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f6d47976fb244a2856edf7b162ffa06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:31 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:46050
I20260929 19:08:31.241226 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f6d47976fb244a2856edf7b162ffa06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:31 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:46056
I20260929 19:08:31.241456 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 9963
I20260929 19:08:31.242547 10401 tablet_replica.cc:330] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: stopping tablet replica
I20260929 19:08:31.242671 10401 raft_consensus.cc:2229] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:31.242790 10401 raft_consensus.cc:2258] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:31.242983 10400 tablet_replica.cc:330] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: stopping tablet replica
I20260929 19:08:31.243077 10400 raft_consensus.cc:2229] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:31.243180 10400 raft_consensus.cc:2258] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:31.243727 10400 ts_tablet_manager.cc:1905] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:31.244731 10401 ts_tablet_manager.cc:1905] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:31.245718 10400 ts_tablet_manager.cc:1918] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:31.245795 10400 log.cc:1196] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/2f6d47976fb244a2856edf7b162ffa06
I20260929 19:08:31.246011 10401 ts_tablet_manager.cc:1918] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:31.246105 10401 log.cc:1196] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/2f6d47976fb244a2856edf7b162ffa06
I20260929 19:08:31.246114 10400 ts_tablet_manager.cc:1939] T 2f6d47976fb244a2856edf7b162ffa06 P d23632b2c390420dbf7d4e8489c35a55: Deleting consensus metadata
I20260929 19:08:31.246356 10401 ts_tablet_manager.cc:1939] T 2f6d47976fb244a2856edf7b162ffa06 P 3f22ed8381304f80bdd36fcb7079be94: Deleting consensus metadata
I20260929 19:08:31.246834 9888 catalog_manager.cc:4841] TS d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383): tablet 2f6d47976fb244a2856edf7b162ffa06 (table test-table [id=bc661177ff4c4a46baafa03f473fb1f6]) successfully deleted
I20260929 19:08:31.246975 9885 catalog_manager.cc:4841] TS 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581): tablet 2f6d47976fb244a2856edf7b162ffa06 (table test-table [id=bc661177ff4c4a46baafa03f473fb1f6]) successfully deleted
W20260929 19:08:31.247239 9885 connection.cc:535] client connection to 127.26.78.193:36697 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 19:08:31.247339 9885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:08:31.247582 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 10097
I20260929 19:08:31.248270 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:31.247697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36697 (local address 127.0.0.1:54270)
0929 19:08:31.248207 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:31.248244 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:36697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490}
W20260929 19:08:31.248404 9885 catalog_manager.cc:4584] TS 51033524a81e41319793b7430f64cb5e (127.26.78.193:36697): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f6d47976fb244a2856edf7b162ffa06: Network error: Client connection negotiation failed: client connection to 127.26.78.193:36697: connect: Connection refused (error 111)
I20260929 19:08:31.253111 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 10231
I20260929 19:08:31.257894 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 9869
I20260929 19:08:51.263533 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40043
--webserver_interface=127.26.78.254
--webserver_port=34437
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:40043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:08:51.345397 10403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:51.345662 10403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:51.345700 10403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:51.346871 10403 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 19:08:51.346948 10403 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 19:08:51.346971 10403 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 19:08:51.346990 10403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:08:51.347018 10403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:51.347050 10403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:51.347074 10403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:08:51.347105 10403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:08:51.348584 10403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:40043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=34437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:51.348866 10403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:51.349359 10403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:51.358788 10412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:51.358907 10403 server_base.cc:1034] running on GCE node
W20260929 19:08:51.358896 10410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:51.358896 10409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:51.359405 10403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:51.359738 10403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:51.360906 10403 hybrid_clock.cc:648] HybridClock initialized: now 1790708931360904 us; error 57 us; skew 500 ppm
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:51.363543 10403 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:08:51.364120 10403 webserver.cc:460] Webserver started at http://127.26.78.254:34437/ using document root <none> and password file <none>
I20260929 19:08:51.364364 10403 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:51.364449 10403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:51.366029 10403 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:51.366705 10419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.366905 10403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:08:51.367012 10403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "79485f4709d94ab89f1dc0d99740cd37"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "72cbdd5f80d6b12207b551bbc1110613"
server_key_iv: "8601863260fa2e34d7a0af8842f1b208"
server_key_version: "encryptionkey@0"
I20260929 19:08:51.367424 10403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:51.396587 10403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:51.399205 10403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:51.399384 10403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:51.403664 10403 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:40043
I20260929 19:08:51.403704 10471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:40043 every 8 connection(s)
I20260929 19:08:51.404145 10403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:08:51.405771 10472 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:08:51.406518 10472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Bootstrap starting.
I20260929 19:08:51.409415 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10403
I20260929 19:08:51.409749 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:36697
--local_ip_for_outbound_sockets=127.26.78.193
--tserver_master_addrs=127.26.78.254:40043
--webserver_port=40137
--webserver_interface=127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:51.415652 10472 log.cc:824] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:51.417651 10472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:08:51.418036 10472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Bootstrap complete.
I20260929 19:08:51.420341 10472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:51.420579 10472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79485f4709d94ab89f1dc0d99740cd37, State: Initialized, Role: FOLLOWER
I20260929 19:08:51.420761 10472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:51.420823 10472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:08:51.420888 10472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:08:51.420941 10472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:08:51.422111 10472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:51.422209 10472 leader_election.cc:302] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79485f4709d94ab89f1dc0d99740cd37; no voters:
I20260929 19:08:51.422435 10472 leader_election.cc:288] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:08:51.422473 10476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:08:51.422636 10476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 LEADER]: Becoming Leader. State: Replica: 79485f4709d94ab89f1dc0d99740cd37, State: Running, Role: LEADER
I20260929 19:08:51.422739 10472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:08:51.422780 10476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:08:51.423062 10476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79485f4709d94ab89f1dc0d99740cd37. Latest consensus state: current_term: 2 leader_uuid: "79485f4709d94ab89f1dc0d99740cd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } } }
I20260929 19:08:51.423130 10476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:51.423239 10477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "79485f4709d94ab89f1dc0d99740cd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } } }
I20260929 19:08:51.423381 10477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: This master's current role is: LEADER
I20260929 19:08:51.424330 10481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:08:51.425227 10481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:08:51.425361 10481 catalog_manager.cc:1259] Loaded cluster ID: 8db1d950e8674e519f70f5d0ff6c3815
I20260929 19:08:51.425423 10481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:08:51.426345 10481 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:08:51.426553 10481 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Loaded TSK: 0
I20260929 19:08:51.436239 10481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Generated new TSK 1
I20260929 19:08:51.436362 10481 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:08:51.508529 10474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:51.508744 10474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:51.508769 10474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:51.510046 10474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:51.510093 10474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:51.510123 10474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:08:51.511622 10474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:36697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=40137
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:51.511879 10474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:51.512125 10474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:51.518177 10500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:51.518214 10499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:51.518190 10502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:51.518442 10474 server_base.cc:1034] running on GCE node
I20260929 19:08:51.518851 10474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:51.519109 10474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:51.520272 10474 hybrid_clock.cc:648] HybridClock initialized: now 1790708931520258 us; error 32 us; skew 500 ppm
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:51.523061 10474 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:08:51.523649 10474 webserver.cc:460] Webserver started at http://127.26.78.193:40137/ using document root <none> and password file <none>
I20260929 19:08:51.523939 10474 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:51.524022 10474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:51.525849 10474 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:08:51.526636 10509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.526886 10474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:08:51.526983 10474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "51033524a81e41319793b7430f64cb5e"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "5fa14328172921aa79019ca436f6a6ea"
server_key_iv: "fe50f906d6aa12f9c57de2c9cb7478d5"
server_key_version: "encryptionkey@0"
I20260929 19:08:51.527436 10474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:51.558775 10474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:51.561275 10474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:51.561441 10474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:51.561728 10474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:51.562286 10516 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:08:51.563834 10474 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:08:51.563898 10474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 19:08:51.563933 10474 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:08:51.564905 10474 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:08:51.564951 10474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:51.565027 10516 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Bootstrap starting.
I20260929 19:08:51.572146 10474 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:36697
I20260929 19:08:51.572674 10474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:08:51.572937 10623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:36697 every 8 connection(s)
I20260929 19:08:51.575999 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10474
I20260929 19:08:51.576108 10516 log.cc:824] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:51.576340 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:38383
--local_ip_for_outbound_sockets=127.26.78.194
--tserver_master_addrs=127.26.78.254:40043
--webserver_port=39337
--webserver_interface=127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:08:51.578960 10516 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:08:51.579295 10516 tablet_bootstrap.cc:492] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Bootstrap complete.
I20260929 19:08:51.580483 10516 ts_tablet_manager.cc:1397] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260929 19:08:51.581929 10516 raft_consensus.cc:348] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.582341 10516 raft_consensus.cc:729] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51033524a81e41319793b7430f64cb5e, State: Initialized, Role: FOLLOWER
I20260929 19:08:51.582561 10516 consensus_queue.cc:260] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.582902 10516 ts_tablet_manager.cc:1428] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:51.589366 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.573649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:42101)
0929 19:08:51.576398 (+ 2749us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.576449 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.578914 (+ 2465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.579494 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.579499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.579879 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:51.580165 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.580171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.581865 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.581865 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.583110 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.583113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.583719 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.583720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.583844 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.584412 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:51.584420 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:51.585763 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:51.587741 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.587748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.587750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.587989 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.588247 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.588248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.588249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.588364 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.588742 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:51.588748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:51.588887 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.588987 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.589163 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2701,"thread_start_us":106,"threads_started":1}
I20260929 19:08:51.589365 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.573784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:42101 (local address 127.26.78.254:40043)
0929 19:08:51.574011 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.574020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.576588 (+ 2568us) server_negotiation.cc:407] Connection header received
0929 19:08:51.579195 (+ 2607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.579203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.579277 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.579445 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:51.580303 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.581709 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.583231 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.583625 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.583747 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.585945 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:51.585955 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:51.585956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:51.585966 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:51.587615 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.588080 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.588083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.588084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.588177 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.588439 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.588440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.588441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.588662 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:51.588749 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.589056 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.589179 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":137,"threads_started":1}
I20260929 19:08:51.590456 10624 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40043
I20260929 19:08:51.590550 10624 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:51.590727 10624 heartbeater.cc:511] Master 127.26.78.254:40043 requested a full tablet report, sending...
I20260929 19:08:51.591537 10436 ts_manager.cc:194] Registered new tserver with Master: 51033524a81e41319793b7430f64cb5e (127.26.78.193:36697)
I20260929 19:08:51.592666 10436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:42101
I20260929 19:08:51.593689 10624 heartbeater.cc:503] Master 127.26.78.254:40043 was elected leader, sending a full tablet report...
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.193@KRBTEST.COM
I20260929 19:08:51.601011 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.592150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36697 (local address 127.0.0.1:47266)
0929 19:08:51.592991 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.593026 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.593085 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.593395 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.593402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.593720 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:51.593780 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.593783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.594857 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.594858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.595472 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.595474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.595940 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.595941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.596016 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.596513 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:51.596519 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:51.597779 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:51.599874 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.599880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.599882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.600092 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.600357 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.600358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.600358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.600418 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.600713 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:51.600719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:51.600760 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.600840 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.600941 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":95,"threads_started":1}
I20260929 19:08:51.601049 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.592391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.26.78.193:36697)
0929 19:08:51.593208 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.593212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.593221 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:08:51.593250 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.593252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.593294 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.593365 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:51.593926 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.594768 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.595560 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.595862 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.595941 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.597939 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:51.597947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:51.597947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:51.597954 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:51.599765 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.600191 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.600194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.600195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.600268 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.600490 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.600491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.600491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.600646 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:51.600727 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.600877 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.600958 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":66,"threads_started":1}
I20260929 19:08:51.601763 10558 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f6d47976fb244a2856edf7b162ffa06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:31 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:47266
I20260929 19:08:51.602353 10633 tablet_replica.cc:330] stopping tablet replica
I20260929 19:08:51.602474 10633 raft_consensus.cc:2229] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:51.602553 10633 raft_consensus.cc:2258] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:51.602870 10633 ts_tablet_manager.cc:1905] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:51.605125 10633 ts_tablet_manager.cc:1918] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:51.605212 10633 log.cc:1196] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal/wals/2f6d47976fb244a2856edf7b162ffa06
I20260929 19:08:51.605449 10633 ts_tablet_manager.cc:1939] T 2f6d47976fb244a2856edf7b162ffa06 P 51033524a81e41319793b7430f64cb5e: Deleting consensus metadata
I20260929 19:08:51.606026 10420 catalog_manager.cc:4841] TS 51033524a81e41319793b7430f64cb5e (127.26.78.193:36697): tablet 2f6d47976fb244a2856edf7b162ffa06 (table test-table [id=bc661177ff4c4a46baafa03f473fb1f6]) successfully deleted
W20260929 19:08:51.665375 10629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:51.665577 10629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:51.665601 10629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:51.666782 10629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:51.666822 10629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:51.666849 10629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:08:51.668303 10629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:38383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=39337
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:51.668555 10629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:51.668792 10629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:51.674021 10641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:51.674021 10639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:51.674136 10638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:51.674136 10629 server_base.cc:1034] running on GCE node
I20260929 19:08:51.674425 10629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:51.674672 10629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:51.675849 10629 hybrid_clock.cc:648] HybridClock initialized: now 1790708931675835 us; error 30 us; skew 500 ppm
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:51.678364 10629 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:08:51.678795 10629 webserver.cc:460] Webserver started at http://127.26.78.194:39337/ using document root <none> and password file <none>
I20260929 19:08:51.679041 10629 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:51.679124 10629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:51.680477 10629 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:51.681032 10648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.681190 10629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.681293 10629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d23632b2c390420dbf7d4e8489c35a55"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "8a0df79d4def1fb89eee0c677601ca60"
server_key_iv: "ce59c0e7754a39e22981c5f0c24763af"
server_key_version: "encryptionkey@0"
I20260929 19:08:51.681639 10629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:51.692730 10629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:51.695142 10629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:51.695309 10629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:51.695544 10629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:51.695852 10629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:51.695907 10629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.695950 10629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:51.695977 10629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.702009 10629 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:38383
I20260929 19:08:51.702085 10761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:38383 every 8 connection(s)
I20260929 19:08:51.702441 10629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:08:51.702916 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10629
I20260929 19:08:51.703186 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:45581
--local_ip_for_outbound_sockets=127.26.78.195
--tserver_master_addrs=127.26.78.254:40043
--webserver_port=43063
--webserver_interface=127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:51.712877 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.703251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:57967)
0929 19:08:51.703431 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.703472 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.703957 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.704207 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.704210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.704906 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:51.705106 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.705111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.706079 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.706080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.706985 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.706987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.707578 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.707579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.707676 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.708234 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:51.708242 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:51.709566 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:51.711563 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.711568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.711570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.711780 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.711996 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.711997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.711997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.712090 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.712382 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:51.712385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:51.712497 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.712575 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.712693 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260929 19:08:51.712883 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.703255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:57967 (local address 127.26.78.254:40043)
0929 19:08:51.703350 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.703353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.703558 (+ 205us) server_negotiation.cc:407] Connection header received
0929 19:08:51.704054 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.704056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.704072 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.704153 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:51.705230 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.705991 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.707088 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.707488 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.707571 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.709673 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:51.709675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:51.709675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:51.709684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:51.711468 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.711854 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.711856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.711856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.711924 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.712159 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.712160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.712160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.712312 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:51.712384 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.712624 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.712733 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 19:08:51.713502 10762 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40043
I20260929 19:08:51.713603 10762 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:51.713788 10762 heartbeater.cc:511] Master 127.26.78.254:40043 requested a full tablet report, sending...
I20260929 19:08:51.714222 10436 ts_manager.cc:194] Registered new tserver with Master: d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
I20260929 19:08:51.714598 10436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:57967
W20260929 19:08:51.786751 10765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:08:51.786973 10765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:08:51.786999 10765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:08:51.788160 10765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:08:51.788203 10765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:08:51.788255 10765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:08:51.789705 10765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:45581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=43063
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:08:51.789975 10765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:08:51.790218 10765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:08:51.795251 10773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:51.795261 10771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:08:51.795274 10770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:08:51.795503 10765 server_base.cc:1034] running on GCE node
I20260929 19:08:51.795680 10765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:08:51.795924 10765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:08:51.797067 10765 hybrid_clock.cc:648] HybridClock initialized: now 1790708931797060 us; error 36 us; skew 500 ppm
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:08:51.799603 10765 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:08:51.800043 10765 webserver.cc:460] Webserver started at http://127.26.78.195:43063/ using document root <none> and password file <none>
I20260929 19:08:51.800271 10765 fs_manager.cc:359] Metadata directory not provided
I20260929 19:08:51.800357 10765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:08:51.801780 10765 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:51.802446 10780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.802609 10765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:08:51.802721 10765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "3f22ed8381304f80bdd36fcb7079be94"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "1159d03b4f0c4f84aa70ff1e8eecb8a8"
server_key_iv: "ee9c9d24a5f94ebde7a6c8e31cbb8f8a"
server_key_version: "encryptionkey@0"
I20260929 19:08:51.803061 10765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:08:51.816972 10765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:08:51.819525 10765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:08:51.819705 10765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:08:51.819950 10765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:08:51.820281 10765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:08:51.820336 10765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.820379 10765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:08:51.820406 10765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:08:51.827039 10765 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:45581
I20260929 19:08:51.827134 10893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:45581 every 8 connection(s)
I20260929 19:08:51.827492 10765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:08:51.828809 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10765
Sep 29 19:08:51 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708931, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:08:51.836738 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.829225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.26.78.254:40043)
0929 19:08:51.829372 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.829374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.829442 (+ 68us) server_negotiation.cc:407] Connection header received
0929 19:08:51.829535 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.829535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.829547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.829609 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:51.830384 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.831084 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.832055 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.832358 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.832429 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.833795 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:51.833797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:51.833797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:51.833804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:51.835436 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.835836 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.835837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.835837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.835894 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.836164 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.836165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.836166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.836355 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:51.836544 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.836587 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.836666 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260929 19:08:51.838893 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.828387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:54969 (local address 127.26.78.254:40043)
0929 19:08:51.828452 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.828455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.828697 (+ 242us) server_negotiation.cc:407] Connection header received
0929 19:08:51.829446 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.829447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.829460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.829526 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:51.830669 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.831315 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.832697 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.832990 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.833069 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.835708 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:51.835709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:51.835709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:51.835715 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:51.837412 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.837957 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.837958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.837958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.838018 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.838351 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.838352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.838352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.838451 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:51.838500 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.838756 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.838833 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:08:51.839102 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.828401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:54969)
0929 19:08:51.828574 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.828610 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.829304 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.829604 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.829609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.830262 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:08:51.830557 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.830564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.831417 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.831418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.832601 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.832604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.833089 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.833090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.833194 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.833703 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:08:51.833711 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:08:51.835620 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:08:51.837486 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.837493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.837495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.837875 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.838149 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:08:51.838150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:08:51.838150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:08:51.838286 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:08:51.838510 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:08:51.838514 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:08:51.838636 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.838716 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.838849 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260929 19:08:51.839536 10894 heartbeater.cc:347] Connected to a master server at 127.26.78.254:40043
I20260929 19:08:51.839622 10894 heartbeater.cc:464] Registering TS with master...
I20260929 19:08:51.839763 10894 heartbeater.cc:511] Master 127.26.78.254:40043 requested a full tablet report, sending...
I20260929 19:08:51.840071 10436 ts_manager.cc:194] Registered new tserver with Master: 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581)
I20260929 19:08:51.840440 10436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:54969
I20260929 19:08:51.849119 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 19:08:51.851913 10365 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.78.254:40043)
I20260929 19:08:51.851967 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.849510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.26.78.254:40043)
0929 19:08:51.849575 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.849577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.849683 (+ 106us) server_negotiation.cc:407] Connection header received
0929 19:08:51.849740 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.849741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.849775 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.849912 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:08:51.850040 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.850501 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.851201 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.851392 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.851440 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.851564 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:08:51.851607 (+ 43us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 19:08:51.851876 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39862: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260929 19:08:51.852133 10627 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39862: token expired
I20260929 19:08:51.859375 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.852407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.26.78.254:40043)
0929 19:08:51.852449 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.852451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.852460 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:08:51.852481 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.852482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.852493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.852558 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:08:51.853035 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.853803 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.854750 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.855076 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.855170 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.856533 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:08:51.856535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:08:51.856535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:08:51.856542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:08:51.858231 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.858674 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.858675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.858675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.858724 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:08:51.858959 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:08:51.858959 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:08:51.858960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:08:51.859054 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:08:51.859128 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.859255 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.859309 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:08:51.860754 10436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:08:51.862474 10436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:08:51.864686 10558 tablet_service.cc:1467] Processing CreateTablet for tablet 6a97135880f34fb2a6a2190cf9837d06 (DEFAULT_TABLE table=test-table [id=4801865b9a82473abb60fa94b8fca6d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:51.864933 10558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a97135880f34fb2a6a2190cf9837d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:51.866524 10516 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Bootstrap starting.
I20260929 19:08:51.866834 10516 tablet_bootstrap.cc:654] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:51.867413 10516 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: No bootstrap required, opened a new log
I20260929 19:08:51.867461 10516 ts_tablet_manager.cc:1397] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:51.867803 10516 raft_consensus.cc:348] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.867889 10516 raft_consensus.cc:374] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:51.867930 10516 raft_consensus.cc:729] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51033524a81e41319793b7430f64cb5e, State: Initialized, Role: FOLLOWER
I20260929 19:08:51.867977 10516 consensus_queue.cc:260] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.868057 10516 ts_tablet_manager.cc:1428] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:08:51.868669 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.864384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:45581 (local address 127.0.0.1:53044)
0929 19:08:51.864419 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.864431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.864468 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.865157 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.865158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.865160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:51.865207 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.865208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.866207 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.866208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.867363 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.867365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.868369 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.868370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.868442 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.868462 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.868548 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.868598 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 19:08:51.868749 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.864766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53044 (local address 127.26.78.195:45581)
0929 19:08:51.864957 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.864978 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.864989 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:08:51.865036 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.865037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.865051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.865138 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:51.865265 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.866123 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.867655 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.868264 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.868367 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.868430 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.868567 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.868610 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260929 19:08:51.868917 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.864764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.26.78.194:38383)
0929 19:08:51.864996 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.865001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.865011 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:08:51.865052 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.865054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.865071 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.865299 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:51.865383 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.866436 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.867651 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.868404 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.868471 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.868529 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.868795 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.868848 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":122,"threads_started":1}
I20260929 19:08:51.869154 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.864458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38383 (local address 127.0.0.1:36634)
0929 19:08:51.864656 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.864666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.864722 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.865183 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.865184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.865186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:51.865231 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.865232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.866525 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.866526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.867582 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.867584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.868607 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.868608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.868676 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.868681 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.868730 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.869096 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":143,"threads_started":1}
I20260929 19:08:51.869429 10828 tablet_service.cc:1467] Processing CreateTablet for tablet 6a97135880f34fb2a6a2190cf9837d06 (DEFAULT_TABLE table=test-table [id=4801865b9a82473abb60fa94b8fca6d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:51.869781 10828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a97135880f34fb2a6a2190cf9837d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:51.869903 10696 tablet_service.cc:1467] Processing CreateTablet for tablet 6a97135880f34fb2a6a2190cf9837d06 (DEFAULT_TABLE table=test-table [id=4801865b9a82473abb60fa94b8fca6d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:08:51.870227 10696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a97135880f34fb2a6a2190cf9837d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:08:51.872632 10904 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Bootstrap starting.
I20260929 19:08:51.872913 10905 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Bootstrap starting.
I20260929 19:08:51.873359 10904 tablet_bootstrap.cc:654] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:51.873569 10905 tablet_bootstrap.cc:654] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Neither blocks nor log segments found. Creating new log.
I20260929 19:08:51.874074 10904 log.cc:824] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:51.874253 10905 log.cc:824] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Log is configured to *not* fsync() on all Append() calls
I20260929 19:08:51.874861 10904 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: No bootstrap required, opened a new log
I20260929 19:08:51.874915 10904 ts_tablet_manager.cc:1397] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:08:51.875018 10905 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: No bootstrap required, opened a new log
I20260929 19:08:51.875097 10905 ts_tablet_manager.cc:1397] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:08:51.876521 10904 raft_consensus.cc:348] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.876663 10904 raft_consensus.cc:374] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:51.876686 10904 raft_consensus.cc:729] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f22ed8381304f80bdd36fcb7079be94, State: Initialized, Role: FOLLOWER
I20260929 19:08:51.876618 10905 raft_consensus.cc:348] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.876803 10905 raft_consensus.cc:374] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:08:51.876847 10905 raft_consensus.cc:729] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23632b2c390420dbf7d4e8489c35a55, State: Initialized, Role: FOLLOWER
I20260929 19:08:51.876813 10904 consensus_queue.cc:260] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.876991 10905 consensus_queue.cc:260] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.877122 10904 ts_tablet_manager.cc:1428] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:08:51.877154 10894 heartbeater.cc:503] Master 127.26.78.254:40043 was elected leader, sending a full tablet report...
I20260929 19:08:51.877269 10905 ts_tablet_manager.cc:1428] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:08:51.877277 10762 heartbeater.cc:503] Master 127.26.78.254:40043 was elected leader, sending a full tablet report...
I20260929 19:08:51.932348 10908 raft_consensus.cc:482] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:51.932513 10908 raft_consensus.cc:504] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.932981 10908 leader_election.cc:288] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383), 51033524a81e41319793b7430f64cb5e (127.26.78.193:36697)
I20260929 19:08:51.935129 10630 raft_consensus.cc:482] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:08:51.935262 10630 raft_consensus.cc:504] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.935609 10630 leader_election.cc:288] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581), d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
I20260929 19:08:51.937083 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.933191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:33691 (local address 127.26.78.193:36697)
0929 19:08:51.933252 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.933254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.933434 (+ 180us) server_negotiation.cc:407] Connection header received
0929 19:08:51.933467 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.933468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.933493 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.933578 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:51.933711 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.934437 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.935979 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.936607 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.936670 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.936742 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.936968 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.937013 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:08:51.937084 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.933124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38383 (local address 127.26.78.195:51057)
0929 19:08:51.933195 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.933208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.933258 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.933426 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.933427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.933429 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:51.933509 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.933511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.934533 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.934534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.935867 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.935870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.936855 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.936856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.936926 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.936930 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.936976 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.937013 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":60}
I20260929 19:08:51.937311 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.933174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36697 (local address 127.26.78.195:33691)
0929 19:08:51.933333 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.933341 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.933420 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.933582 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.933583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.933585 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:51.933647 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.933648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.934533 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.934534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.935867 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.935870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.936804 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.936805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.936884 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.936887 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.937198 (+ 311us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.937243 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":40,"thread_start_us":95,"threads_started":1}
I20260929 19:08:51.937357 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.933243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:51057 (local address 127.26.78.194:38383)
0929 19:08:51.933291 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.933294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.933301 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:08:51.933338 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.933339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.933346 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.933396 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:51.933585 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.934401 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.936051 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.936633 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.936691 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.936700 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.937248 (+ 548us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.937286 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:08:51.937644 10716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a97135880f34fb2a6a2190cf9837d06" candidate_uuid: "3f22ed8381304f80bdd36fcb7079be94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d23632b2c390420dbf7d4e8489c35a55" is_pre_election: true
I20260929 19:08:51.937747 10578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a97135880f34fb2a6a2190cf9837d06" candidate_uuid: "3f22ed8381304f80bdd36fcb7079be94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51033524a81e41319793b7430f64cb5e" is_pre_election: true
I20260929 19:08:51.937788 10716 raft_consensus.cc:2454] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f22ed8381304f80bdd36fcb7079be94 in term 0.
I20260929 19:08:51.937856 10578 raft_consensus.cc:2454] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f22ed8381304f80bdd36fcb7079be94 in term 0.
I20260929 19:08:51.938041 10784 leader_election.cc:302] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f22ed8381304f80bdd36fcb7079be94, d23632b2c390420dbf7d4e8489c35a55; no voters:
I20260929 19:08:51.938227 10908 raft_consensus.cc:2781] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:08:51.938304 10908 raft_consensus.cc:482] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:08:51.938331 10908 raft_consensus.cc:3037] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:51.939230 10908 raft_consensus.cc:504] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.939489 10908 leader_election.cc:288] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [CANDIDATE]: Term 1 election: Requested vote from peers d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383), 51033524a81e41319793b7430f64cb5e (127.26.78.193:36697)
I20260929 19:08:51.939503 10716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a97135880f34fb2a6a2190cf9837d06" candidate_uuid: "3f22ed8381304f80bdd36fcb7079be94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d23632b2c390420dbf7d4e8489c35a55"
I20260929 19:08:51.939543 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.935745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:45581 (local address 127.26.78.193:43353)
0929 19:08:51.935926 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.935937 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.935981 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.936251 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.936252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.936254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:51.936327 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.936328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.937322 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.937323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.938759 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.938760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.939274 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.939275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.939357 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.939361 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.939410 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.939454 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260929 19:08:51.939549 10578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a97135880f34fb2a6a2190cf9837d06" candidate_uuid: "3f22ed8381304f80bdd36fcb7079be94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51033524a81e41319793b7430f64cb5e"
I20260929 19:08:51.939591 10716 raft_consensus.cc:3037] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:51.939653 10578 raft_consensus.cc:3037] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:08:51.939815 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.936010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:43353 (local address 127.26.78.195:45581)
0929 19:08:51.936114 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.936116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.936125 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:08:51.936147 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.936148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.936158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.936209 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:51.936402 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.937115 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.938840 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.939189 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.939275 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.939283 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.939708 (+ 425us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.939752 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 19:08:51.939862 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.935751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38383 (local address 127.26.78.193:44035)
0929 19:08:51.935933 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.935943 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:08:51.935981 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:08:51.936605 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:08:51.936605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:08:51.936607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:08:51.936650 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.936651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.937503 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.937504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.938672 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:08:51.938674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.939581 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.939582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:08:51.939643 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.939646 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:08:51.939700 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 19:08:51.939743 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":11,"thread_start_us":123,"threads_started":1}
I20260929 19:08:51.940054 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.936094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:44035 (local address 127.26.78.194:38383)
0929 19:08:51.936469 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.936471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.936478 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:08:51.936497 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.936498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.936506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.936565 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:08:51.936715 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.937425 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.938762 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.939360 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.939438 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.939446 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.939945 (+ 499us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.939989 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":109,"threads_started":1}
I20260929 19:08:51.940026 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a97135880f34fb2a6a2190cf9837d06" candidate_uuid: "51033524a81e41319793b7430f64cb5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f22ed8381304f80bdd36fcb7079be94" is_pre_election: true
I20260929 19:08:51.940121 10848 raft_consensus.cc:2379] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51033524a81e41319793b7430f64cb5e in current term 1: Already voted for candidate 3f22ed8381304f80bdd36fcb7079be94 in this term.
I20260929 19:08:51.940541 10716 raft_consensus.cc:2454] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f22ed8381304f80bdd36fcb7079be94 in term 1.
I20260929 19:08:51.940636 10578 raft_consensus.cc:2454] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f22ed8381304f80bdd36fcb7079be94 in term 1.
I20260929 19:08:51.940716 10784 leader_election.cc:302] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f22ed8381304f80bdd36fcb7079be94, d23632b2c390420dbf7d4e8489c35a55; no voters:
I20260929 19:08:51.940750 10715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a97135880f34fb2a6a2190cf9837d06" candidate_uuid: "51033524a81e41319793b7430f64cb5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d23632b2c390420dbf7d4e8489c35a55" is_pre_election: true
I20260929 19:08:51.940796 10908 raft_consensus.cc:2781] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:08:51.940830 10715 raft_consensus.cc:2379] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51033524a81e41319793b7430f64cb5e in current term 1: Already voted for candidate 3f22ed8381304f80bdd36fcb7079be94 in this term.
I20260929 19:08:51.940949 10908 raft_consensus.cc:686] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 LEADER]: Becoming Leader. State: Replica: 3f22ed8381304f80bdd36fcb7079be94, State: Running, Role: LEADER
I20260929 19:08:51.941044 10908 consensus_queue.cc:237] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:08:51.941143 10513 leader_election.cc:302] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51033524a81e41319793b7430f64cb5e; no voters: 3f22ed8381304f80bdd36fcb7079be94, d23632b2c390420dbf7d4e8489c35a55
I20260929 19:08:51.941293 10630 raft_consensus.cc:2726] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 19:08:51.942018 10435 catalog_manager.cc:5495] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "3f22ed8381304f80bdd36fcb7079be94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } health_report { overall_health: UNKNOWN } } }
I20260929 19:08:51.947710 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.945423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.26.78.195:45581)
0929 19:08:51.945475 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:08:51.945476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:08:51.945483 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:08:51.945516 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:08:51.945517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:08:51.945530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:08:51.945608 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:08:51.945710 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.946234 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.946951 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:08:51.947152 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:08:51.947235 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:08:51.947375 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:08:51.947479 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:08:51.947572 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 19:08:51.947591 (+ 19us) server_negotiation.cc:299] Negotiation successful
0929 19:08:51.947609 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:08:51.949352 10715 raft_consensus.cc:1261] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Refusing update from remote peer 3f22ed8381304f80bdd36fcb7079be94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:51.949417 10578 raft_consensus.cc:1261] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Refusing update from remote peer 3f22ed8381304f80bdd36fcb7079be94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:08:51.949663 10908 consensus_queue.cc:1035] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:51.949859 10913 consensus_queue.cc:1035] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:08:51.951884 10916 mvcc.cc:205] Tried to move back new op lower bound from 7334743785262030848 to 7334743785231474688. Current Snapshot: MvccSnapshot[applied={T|T < 7334743785262030848}]
I20260929 19:08:51.952122 10918 mvcc.cc:205] Tried to move back new op lower bound from 7334743785262030848 to 7334743785231474688. Current Snapshot: MvccSnapshot[applied={T|T < 7334743785262030848}]
I20260929 19:08:51.952458 10917 mvcc.cc:205] Tried to move back new op lower bound from 7334743785262030848 to 7334743785231474688. Current Snapshot: MvccSnapshot[applied={T|T < 7334743785262030848}]
I20260929 19:08:51.954810 10435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:51.954906 10435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:08:51.955590 10435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Sending DeleteTablet for 3 replicas of tablet 6a97135880f34fb2a6a2190cf9837d06
I20260929 19:08:51.955905 10558 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a97135880f34fb2a6a2190cf9837d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:51 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:47266
I20260929 19:08:51.955888 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a97135880f34fb2a6a2190cf9837d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:51 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:53044
I20260929 19:08:51.955924 10696 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a97135880f34fb2a6a2190cf9837d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:08:51 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:36634
I20260929 19:08:51.956007 10633 tablet_replica.cc:330] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: stopping tablet replica
I20260929 19:08:51.956080 10633 raft_consensus.cc:2229] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:51.956136 10633 raft_consensus.cc:2258] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:51.956210 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 10474
I20260929 19:08:51.956885 10926 tablet_replica.cc:330] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: stopping tablet replica
I20260929 19:08:51.956988 10926 raft_consensus.cc:2229] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:08:51.957108 10926 raft_consensus.cc:2258] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:51.957242 10927 tablet_replica.cc:330] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: stopping tablet replica
I20260929 19:08:51.957331 10927 raft_consensus.cc:2229] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:08:51.957429 10927 raft_consensus.cc:2258] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:08:51.957916 10927 ts_tablet_manager.cc:1905] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:08:51.959919 10927 ts_tablet_manager.cc:1918] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:51.959983 10927 log.cc:1196] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/6a97135880f34fb2a6a2190cf9837d06
I20260929 19:08:51.960588 10927 ts_tablet_manager.cc:1939] T 6a97135880f34fb2a6a2190cf9837d06 P d23632b2c390420dbf7d4e8489c35a55: Deleting consensus metadata
I20260929 19:08:51.961089 10423 catalog_manager.cc:4841] TS d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383): tablet 6a97135880f34fb2a6a2190cf9837d06 (table test-table [id=4801865b9a82473abb60fa94b8fca6d6]) successfully deleted
I20260929 19:08:51.962896 10926 ts_tablet_manager.cc:1905] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:08:51.963654 10420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:36697 (error 108)
I20260929 19:08:51.963867 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 10629
I20260929 19:08:51.964325 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:08:51.964209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:36697 (local address 127.0.0.1:47280)
0929 19:08:51.964254 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:08:51.964284 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:36697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:08:51.964664 10926 ts_tablet_manager.cc:1918] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:08:51.964737 10926 log.cc:1196] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/6a97135880f34fb2a6a2190cf9837d06
I20260929 19:08:51.965055 10926 ts_tablet_manager.cc:1939] T 6a97135880f34fb2a6a2190cf9837d06 P 3f22ed8381304f80bdd36fcb7079be94: Deleting consensus metadata
W20260929 19:08:51.964584 10420 catalog_manager.cc:4584] TS 51033524a81e41319793b7430f64cb5e (127.26.78.193:36697): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a97135880f34fb2a6a2190cf9837d06: Network error: Client connection negotiation failed: client connection to 127.26.78.193:36697: connect: Connection refused (error 111)
I20260929 19:08:51.965698 10420 catalog_manager.cc:4841] TS 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581): tablet 6a97135880f34fb2a6a2190cf9837d06 (table test-table [id=4801865b9a82473abb60fa94b8fca6d6]) successfully deleted
I20260929 19:08:51.969713 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 10765
I20260929 19:08:51.974828 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 10403
I20260929 19:09:11.980330 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40043
--webserver_interface=127.26.78.254
--webserver_port=34437
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:40043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:12.061389 10929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:12.061599 10929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:12.061622 10929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:12.062844 10929 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 19:09:12.062889 10929 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 19:09:12.062902 10929 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 19:09:12.062914 10929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:12.062927 10929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:12.062940 10929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:12.062952 10929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:12.063012 10929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:12.064603 10929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:40043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:40043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=34437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:12.064865 10929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:12.065155 10929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:12.070564 10937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:12.070626 10929 server_base.cc:1034] running on GCE node
W20260929 19:09:12.070580 10934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:12.070581 10935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:12.071077 10929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:12.071396 10929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:12.072569 10929 hybrid_clock.cc:648] HybridClock initialized: now 1790708952072560 us; error 37 us; skew 500 ppm
Sep 29 19:09:12 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708952, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:12.075171 10929 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:12.075639 10929 webserver.cc:460] Webserver started at http://127.26.78.254:34437/ using document root <none> and password file <none>
I20260929 19:09:12.075898 10929 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:12.075947 10929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:12.077296 10929 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:12.078065 10944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.078276 10929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:12.078370 10929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "79485f4709d94ab89f1dc0d99740cd37"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "72cbdd5f80d6b12207b551bbc1110613"
server_key_iv: "8601863260fa2e34d7a0af8842f1b208"
server_key_version: "encryptionkey@0"
I20260929 19:09:12.078718 10929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:12.088218 10929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:12.090225 10929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:12.090381 10929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:12.094583 10929 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:40043
I20260929 19:09:12.094622 10996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:40043 every 8 connection(s)
I20260929 19:09:12.095031 10929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:12.095533 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10929
I20260929 19:09:12.095868 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:36697
--local_ip_for_outbound_sockets=127.26.78.193
--tserver_master_addrs=127.26.78.254:40043
--webserver_port=40137
--webserver_interface=127.26.78.193
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:09:12.097247 10997 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:09:12.097992 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Bootstrap starting.
I20260929 19:09:12.106323 10997 log.cc:824] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:12.109023 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:09:12.109364 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Bootstrap complete.
I20260929 19:09:12.111977 10997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:09:12.112228 10997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79485f4709d94ab89f1dc0d99740cd37, State: Initialized, Role: FOLLOWER
I20260929 19:09:12.112416 10997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:09:12.112478 10997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:12.112525 10997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:12.112574 10997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 2 FOLLOWER]: Advancing to term 3
I20260929 19:09:12.113560 10997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:09:12.113642 10997 leader_election.cc:302] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79485f4709d94ab89f1dc0d99740cd37; no voters:
I20260929 19:09:12.113875 10997 leader_election.cc:288] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 19:09:12.113911 11001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 19:09:12.114019 11001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [term 3 LEADER]: Becoming Leader. State: Replica: 79485f4709d94ab89f1dc0d99740cd37, State: Running, Role: LEADER
I20260929 19:09:12.114102 11001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } }
I20260929 19:09:12.114143 10997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:12.114367 11002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "79485f4709d94ab89f1dc0d99740cd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } } }
I20260929 19:09:12.114451 11002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:12.114362 11001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79485f4709d94ab89f1dc0d99740cd37. Latest consensus state: current_term: 3 leader_uuid: "79485f4709d94ab89f1dc0d99740cd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79485f4709d94ab89f1dc0d99740cd37" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 40043 } } }
I20260929 19:09:12.114632 11001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:12.114764 11005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:12.115614 11005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:12.115753 11005 catalog_manager.cc:1259] Loaded cluster ID: 8db1d950e8674e519f70f5d0ff6c3815
I20260929 19:09:12.115947 11005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:12.116744 11005 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:12.116953 11005 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Loaded TSK: 0
I20260929 19:09:12.117012 11005 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79485f4709d94ab89f1dc0d99740cd37: Loaded TSK: 1
F20260929 19:09:12.117234 11005 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f33a84949ad google::LogMessage::Fail() at ??:0
@ 0x7f33a849788a google::LogMessage::SendToLog() at ??:0
@ 0x7f33a84944de google::LogMessage::Flush() at ??:0
@ 0x7f33a8495a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f33aac01104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f33aabfed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f33b03ee1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f33b03ee48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f33b03ee594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f33b03b4c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f33b03bead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f33aa3620d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f33aa357954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f33ab1bb6db start_thread at ??:0
@ 0x7f33a6f9261f clone at ??:0
W20260929 19:09:12.192834 10999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:12.193066 10999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:12.193109 10999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:12.194347 10999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:12.194408 10999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:12.194453 10999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:09:12.195920 10999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:36697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=40137
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:12.196190 10999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:12.196426 10999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:12.202489 11039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:12.202524 11038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:12.202768 10999 server_base.cc:1034] running on GCE node
W20260929 19:09:12.202504 11041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:12.203081 10999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:12.203332 10999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:12.204509 10999 hybrid_clock.cc:648] HybridClock initialized: now 1790708952204495 us; error 30 us; skew 500 ppm
Sep 29 19:09:12 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708952, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:12.207270 10999 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:09:12.207772 10999 webserver.cc:460] Webserver started at http://127.26.78.193:40137/ using document root <none> and password file <none>
I20260929 19:09:12.208017 10999 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:12.208084 10999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:12.209434 10999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:12.210130 11048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.210326 10999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:12.210430 10999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "51033524a81e41319793b7430f64cb5e"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "5fa14328172921aa79019ca436f6a6ea"
server_key_iv: "fe50f906d6aa12f9c57de2c9cb7478d5"
server_key_version: "encryptionkey@0"
I20260929 19:09:12.210889 10999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:12.225422 10999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:12.228060 10999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:12.228238 10999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:12.228523 10999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:12.229043 11055 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:09:12.230607 10999 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:09:12.230666 10999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 19:09:12.230700 10999 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:09:12.231699 10999 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:09:12.231743 10999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 19:09:12.231830 11055 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Bootstrap starting.
I20260929 19:09:12.239035 10999 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:36697
I20260929 19:09:12.239239 11162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:36697 every 8 connection(s)
I20260929 19:09:12.239511 10999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:09:12.241405 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 10999
I20260929 19:09:12.241725 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:38383
--local_ip_for_outbound_sockets=127.26.78.194
--tserver_master_addrs=127.26.78.254:40043
--webserver_port=39337
--webserver_interface=127.26.78.194
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:09:12.242504 11055 log.cc:824] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:12.245003 11055 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:09:12.245340 11055 tablet_bootstrap.cc:492] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Bootstrap complete.
I20260929 19:09:12.246513 11055 ts_tablet_manager.cc:1397] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
I20260929 19:09:12.248315 11055 raft_consensus.cc:348] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:09:12.248735 11055 raft_consensus.cc:729] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51033524a81e41319793b7430f64cb5e, State: Initialized, Role: FOLLOWER
I20260929 19:09:12.248926 11055 consensus_queue.cc:260] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:09:12.249257 11055 ts_tablet_manager.cc:1428] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260929 19:09:12.324460 11165 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:09:12.240280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:33705)
0929 19:09:12.240503 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.240558 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:12.241388 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:12.324257 (+ 82869us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":103,"threads_started":1}
W20260929 19:09:12.324738 11163 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:40043 (0 consecutive failures): Network error: Failed to ping master at 127.26.78.254:40043: Client connection negotiation failed: client connection to 127.26.78.254:40043: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:09:12.325122 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.325008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:58355)
0929 19:09:12.325061 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.325101 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 19:09:12.325506 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.325403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:41329)
0929 19:09:12.325448 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.325471 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 19:09:12.325713 11163 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:09:12.327555 11167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:12.327831 11167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:12.327880 11167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:12.329041 11167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:12.329106 11167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:12.329152 11167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:09:12.330673 11167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:38383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=39337
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:12.330952 11167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:12.331177 11167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:12.336354 11173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:12.336364 11174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:12.336545 11167 server_base.cc:1034] running on GCE node
W20260929 19:09:12.336383 11176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:12.336789 11167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:12.337069 11167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:12.338213 11167 hybrid_clock.cc:648] HybridClock initialized: now 1790708952338199 us; error 27 us; skew 500 ppm
Sep 29 19:09:12 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708952, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:12.340734 11167 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:09:12.341210 11167 webserver.cc:460] Webserver started at http://127.26.78.194:39337/ using document root <none> and password file <none>
I20260929 19:09:12.341449 11167 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:12.341518 11167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:12.343004 11167 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:12.343609 11183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.343777 11167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:12.343886 11167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d23632b2c390420dbf7d4e8489c35a55"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "8a0df79d4def1fb89eee0c677601ca60"
server_key_iv: "ce59c0e7754a39e22981c5f0c24763af"
server_key_version: "encryptionkey@0"
I20260929 19:09:12.344215 11167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:12.365176 11167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:12.367632 11167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:12.367805 11167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:12.368058 11167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:12.368376 11167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:12.368435 11167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.368470 11167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:12.368497 11167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.374645 11167 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:38383
I20260929 19:09:12.374691 11296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:38383 every 8 connection(s)
I20260929 19:09:12.375046 11167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:09:12.376039 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.375671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:36623)
0929 19:09:12.375820 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.375900 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
W20260929 19:09:12.376441 11297 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:40043 (0 consecutive failures): Network error: Failed to ping master at 127.26.78.254:40043: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
I20260929 19:09:12.376729 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.376642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:60535)
0929 19:09:12.376687 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.376712 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 19:09:12.376995 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.376947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:60965)
0929 19:09:12.376966 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.376983 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260929 19:09:12.377116 11297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:09:12.378643 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 11167
I20260929 19:09:12.378942 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:45581
--local_ip_for_outbound_sockets=127.26.78.195
--tserver_master_addrs=127.26.78.254:40043
--webserver_port=43063
--webserver_interface=127.26.78.195
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:12.461741 11300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:12.461987 11300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:12.462013 11300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:12.463115 11300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:12.463152 11300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:12.463177 11300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:09:12.464839 11300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:45581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=43063
--tserver_master_addrs=127.26.78.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:12.465034 11300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:12.465310 11300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:12.470332 11305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:12.470304 11306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:12.470352 11308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:12.470502 11300 server_base.cc:1034] running on GCE node
I20260929 19:09:12.470711 11300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:12.470966 11300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:12.472111 11300 hybrid_clock.cc:648] HybridClock initialized: now 1790708952472093 us; error 35 us; skew 500 ppm
Sep 29 19:09:12 dist-test-slave-xz51 krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708952, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:12.474786 11300 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:09:12.475276 11300 webserver.cc:460] Webserver started at http://127.26.78.195:43063/ using document root <none> and password file <none>
I20260929 19:09:12.475544 11300 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:12.475593 11300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:12.476962 11300 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:12.477579 11315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.477761 11300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:12.477885 11300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "3f22ed8381304f80bdd36fcb7079be94"
format_stamp: "Formatted at 2026-09-29 19:08:30 on dist-test-slave-xz51"
server_key: "1159d03b4f0c4f84aa70ff1e8eecb8a8"
server_key_iv: "ee9c9d24a5f94ebde7a6c8e31cbb8f8a"
server_key_version: "encryptionkey@0"
I20260929 19:09:12.478223 11300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:12.496325 11300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:12.498319 11300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:12.498492 11300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:12.498764 11300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:12.499090 11300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:12.499146 11300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.499187 11300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:12.499215 11300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:12.505069 11300 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:45581
I20260929 19:09:12.505119 11428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:45581 every 8 connection(s)
I20260929 19:09:12.505468 11300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:09:12.506527 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.506106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:38547)
0929 19:09:12.506256 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.506376 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
W20260929 19:09:12.506948 11429 heartbeater.cc:644] Failed to heartbeat to 127.26.78.254:40043 (0 consecutive failures): Network error: Failed to ping master at 127.26.78.254:40043: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
I20260929 19:09:12.507310 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.507163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:60129)
0929 19:09:12.507232 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.507287 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 19:09:12.507834 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:12.507682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:46683)
0929 19:09:12.507750 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:12.507784 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 19:09:12.507953 11429 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:09:12.514214 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 11300
I20260929 19:09:13.326262 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:13.326059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:48839)
0929 19:09:13.326170 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:13.326201 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260929 19:09:13.377813 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:13.377485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:52693)
0929 19:09:13.377698 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:13.377746 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":51,"threads_started":1}
I20260929 19:09:13.508545 11435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:13.508324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:43703)
0929 19:09:13.508457 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:13.508494 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":56,"threads_started":1}
I20260929 19:09:13.726584 11436 raft_consensus.cc:482] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:13.726738 11436 raft_consensus.cc:504] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:09:13.727085 11436 leader_election.cc:288] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581), d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
Sep 29 19:09:13 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708952, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.194@KRBTEST.COM
Sep 29 19:09:13 dist-test-slave-xz51 krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708952, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.195@KRBTEST.COM
I20260929 19:09:13.787843 11438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:13.727438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:47737 (local address 127.26.78.195:45581)
0929 19:09:13.727585 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:13.727598 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:13.727632 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:09:13.728194 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:13.728197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:13.728257 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:13.728414 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:13.729416 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.730854 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.732410 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.733037 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.733154 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:13.783147 (+ 49993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:13.783153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:13.783154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:13.783169 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:13.785467 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:13.786225 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:13.786227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:13.786228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:13.786347 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:13.786703 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:13.786704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:13.786704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:13.787095 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:13.787189 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:13.787578 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 19:09:13.787726 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":47,"threads_started":1}
I20260929 19:09:13.787844 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:13.727276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:45581 (local address 127.26.78.193:47737)
0929 19:09:13.727335 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:13.727347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:13.727417 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:13.728493 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:13.728496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:13.728996 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:13.729281 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:13.729288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.730971 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.730971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:13.732284 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:13.732287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.733140 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.733141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:13.733276 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:13.780538 (+ 47262us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:13.780553 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:13.783003 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:13.785577 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:13.785589 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:13.785592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:13.786082 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:13.786435 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:13.786436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:13.786436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:13.786605 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:13.787181 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:13.787189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:13.787364 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 19:09:13.787490 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 19:09:13.787714 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":158}
I20260929 19:09:13.787869 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:13.727455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:40941 (local address 127.26.78.194:38383)
0929 19:09:13.727585 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:13.727598 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:13.727632 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:09:13.728194 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:13.728197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:13.728257 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:13.728413 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:13.729416 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.730854 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.732410 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.733037 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.733154 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:13.783172 (+ 50018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:13.783176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:13.783176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:13.783187 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:13.785463 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:13.786212 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:13.786216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:13.786217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:13.786347 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:13.786703 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:13.786704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:13.786704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:13.787095 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:13.787189 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:13.787578 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 19:09:13.787725 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260929 19:09:13.787853 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:13.727296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:38383 (local address 127.26.78.193:40941)
0929 19:09:13.727415 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:13.727426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:13.727480 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:13.728493 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:13.728496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:13.728996 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:13.729281 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:13.729288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.730970 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.730971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:13.732284 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:13.732287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:13.733141 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:13.733142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:13.733276 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:13.780538 (+ 47262us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:13.780570 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:13.782681 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:13.785577 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:13.785589 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:13.785592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:13.786082 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:13.786435 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:13.786436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:13.786436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:13.786605 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:13.787181 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:13.787189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:13.787364 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 19:09:13.787490 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 19:09:13.787768 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":163,"thread_start_us":62,"threads_started":1}
W20260929 19:09:13.788990 11052 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
W20260929 19:09:13.789256 11049 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
I20260929 19:09:13.789356 11049 leader_election.cc:302] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51033524a81e41319793b7430f64cb5e; no voters: 3f22ed8381304f80bdd36fcb7079be94, d23632b2c390420dbf7d4e8489c35a55
I20260929 19:09:13.789522 11436 raft_consensus.cc:2726] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:09:14.326998 11440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:14.326736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:33499)
0929 19:09:14.326871 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:14.326915 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260929 19:09:14.378516 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:14.378312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:39117)
0929 19:09:14.378441 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:14.378479 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260929 19:09:14.509308 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:14.509030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:41877)
0929 19:09:14.509204 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:14.509263 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":66,"threads_started":1}
I20260929 19:09:15.327783 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:15.327542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:38821)
0929 19:09:15.327699 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:15.327736 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260929 19:09:15.379174 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:15.378954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:50185)
0929 19:09:15.379080 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:15.379134 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260929 19:09:15.425477 11445 raft_consensus.cc:482] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:15.425580 11445 raft_consensus.cc:504] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:09:15.425788 11445 leader_election.cc:288] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581), d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
W20260929 19:09:15.426282 11049 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
W20260929 19:09:15.426359 11052 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
I20260929 19:09:15.426450 11052 leader_election.cc:302] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51033524a81e41319793b7430f64cb5e; no voters: 3f22ed8381304f80bdd36fcb7079be94, d23632b2c390420dbf7d4e8489c35a55
I20260929 19:09:15.426556 11445 raft_consensus.cc:2726] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:09:15.510015 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:15.509832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:51641)
0929 19:09:15.509934 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:15.509972 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":57,"threads_started":1}
I20260929 19:09:16.328608 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:16.328326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:44525)
0929 19:09:16.328513 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:16.328561 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
I20260929 19:09:16.379969 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:16.379740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:32777)
0929 19:09:16.379887 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:16.379925 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260929 19:09:16.510660 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:16.510453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:34405)
0929 19:09:16.510588 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:16.510618 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260929 19:09:17.329550 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:17.329325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:37311)
0929 19:09:17.329458 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:17.329503 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260929 19:09:17.380676 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:17.380431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:35479)
0929 19:09:17.380593 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:17.380633 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":55,"threads_started":1}
I20260929 19:09:17.511322 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:17.511100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:40023)
0929 19:09:17.511230 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:17.511275 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260929 19:09:18.330318 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:18.330093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:50397)
0929 19:09:18.330230 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:18.330271 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260929 19:09:18.381485 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:18.381186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:46473)
0929 19:09:18.381362 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:18.381413 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260929 19:09:18.450924 11455 raft_consensus.cc:482] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:18.451022 11455 raft_consensus.cc:504] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:09:18.451331 11455 leader_election.cc:288] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581), d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
W20260929 19:09:18.451733 11052 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
W20260929 19:09:18.451822 11049 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
I20260929 19:09:18.451900 11049 leader_election.cc:302] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51033524a81e41319793b7430f64cb5e; no voters: 3f22ed8381304f80bdd36fcb7079be94, d23632b2c390420dbf7d4e8489c35a55
I20260929 19:09:18.451967 11455 raft_consensus.cc:2726] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:09:18.512084 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:18.511855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:43981)
0929 19:09:18.511995 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:18.512039 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260929 19:09:19.330952 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:19.330753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:40279)
0929 19:09:19.330871 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:19.330907 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260929 19:09:19.382306 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:19.382017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:59947)
0929 19:09:19.382218 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:19.382261 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":102,"threads_started":1}
I20260929 19:09:19.512828 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:19.512591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:57381)
0929 19:09:19.512736 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:19.512778 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260929 19:09:20.331749 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:20.331505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:57493)
0929 19:09:20.331655 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:20.331701 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260929 19:09:20.383064 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:20.382823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:56269)
0929 19:09:20.382980 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:20.383018 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260929 19:09:20.513562 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:20.513314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:52745)
0929 19:09:20.513471 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:20.513515 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260929 19:09:21.332384 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:21.332164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:52659)
0929 19:09:21.332295 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:21.332332 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260929 19:09:21.383769 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:21.383510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:36937)
0929 19:09:21.383643 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:21.383679 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260929 19:09:21.514305 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:21.514090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:60323)
0929 19:09:21.514214 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:21.514255 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260929 19:09:21.896796 11466 raft_consensus.cc:482] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:21.896914 11466 raft_consensus.cc:504] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22ed8381304f80bdd36fcb7079be94" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 45581 } } peers { permanent_uuid: "d23632b2c390420dbf7d4e8489c35a55" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 38383 } } peers { permanent_uuid: "51033524a81e41319793b7430f64cb5e" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 36697 } }
I20260929 19:09:21.897053 11466 leader_election.cc:288] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581), d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383)
W20260929 19:09:21.897540 11049 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f22ed8381304f80bdd36fcb7079be94 (127.26.78.195:45581): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
W20260929 19:09:21.897630 11052 leader_election.cc:341] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d23632b2c390420dbf7d4e8489c35a55 (127.26.78.194:38383): Not found: Tablet not found: 6a97135880f34fb2a6a2190cf9837d06
I20260929 19:09:21.897733 11052 leader_election.cc:302] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51033524a81e41319793b7430f64cb5e; no voters: 3f22ed8381304f80bdd36fcb7079be94, d23632b2c390420dbf7d4e8489c35a55
I20260929 19:09:21.897845 11466 raft_consensus.cc:2726] T 6a97135880f34fb2a6a2190cf9837d06 P 51033524a81e41319793b7430f64cb5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:09:22.333273 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:22.333012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:39405)
0929 19:09:22.333180 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:22.333226 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":67,"threads_started":1}
I20260929 19:09:22.384593 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:22.384357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:59191)
0929 19:09:22.384507 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:22.384549 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260929 19:09:22.515107 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:22.514875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:56029)
0929 19:09:22.515017 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:22.515058 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260929 19:09:23.334101 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:23.333789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:58259)
0929 19:09:23.334012 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:23.334055 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":119,"threads_started":1}
I20260929 19:09:23.385278 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:23.385064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:59151)
0929 19:09:23.385192 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:23.385231 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260929 19:09:23.515829 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:23.515554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:54707)
0929 19:09:23.515694 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:23.515735 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260929 19:09:24.334857 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:24.334609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:33559)
0929 19:09:24.334744 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:24.334786 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260929 19:09:24.386075 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:24.385858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:38271)
0929 19:09:24.385992 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:24.386028 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260929 19:09:24.516516 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:24.516298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:51605)
0929 19:09:24.516430 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:24.516471 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260929 19:09:25.335557 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:25.335316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:39237)
0929 19:09:25.335467 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:25.335509 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260929 19:09:25.386736 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:25.386542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:49283)
0929 19:09:25.386651 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:25.386688 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260929 19:09:25.517244 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:25.517007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:52103)
0929 19:09:25.517161 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:25.517196 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260929 19:09:26.336299 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:26.336056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:36029)
0929 19:09:26.336213 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:26.336253 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260929 19:09:26.387425 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:26.387186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:57701)
0929 19:09:26.387322 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:26.387378 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260929 19:09:26.518003 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:26.517744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:37847)
0929 19:09:26.517914 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:26.517955 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":126,"threads_started":1}
I20260929 19:09:27.337057 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:27.336815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.193:45909)
0929 19:09:27.336960 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:27.336995 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260929 19:09:27.388273 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:27.388053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.194:50641)
0929 19:09:27.388187 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:27.388226 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260929 19:09:27.515647 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 10999
I20260929 19:09:27.518813 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:27.518583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:40043 (local address 127.26.78.195:58461)
0929 19:09:27.518725 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:27.518768 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.254:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260929 19:09:27.520938 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 11167
I20260929 19:09:27.525182 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 11300
2026-09-29T19:09:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58491 ms)
[----------] 30 tests from SecurityITest (257327 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:27 dist-test-slave-xz51 krb5kdc[11487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:27 dist-test-slave-xz51 krb5kdc[11487](info): set up 2 sockets
Sep 29 19:09:27 dist-test-slave-xz51 krb5kdc[11487](info): commencing operation
krb5kdc: starting...
W20260929 19:09:28.574388 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:28 dist-test-slave-xz51 krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:28.697152 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:45953
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:45953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:28.779182 11500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:28.779467 11500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:28.779505 11500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:28.780681 11500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:28.780753 11500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:28.780778 11500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:28.780812 11500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:28.780838 11500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:28.782445 11500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:45953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:45953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:28.782744 11500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:28.782996 11500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:28.788570 11506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:28.788661 11500 server_base.cc:1034] running on GCE node
W20260929 19:09:28.788601 11508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:28.788575 11505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:28.789055 11500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:28.789363 11500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:28.790532 11500 hybrid_clock.cc:648] HybridClock initialized: now 1790708968790517 us; error 28 us; skew 500 ppm
Sep 29 19:09:28 dist-test-slave-xz51 krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708968, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:28.793152 11500 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:28.793609 11500 webserver.cc:460] Webserver started at http://127.26.78.254:35197/ using document root <none> and password file <none>
I20260929 19:09:28.793897 11500 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:28.793967 11500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:28.794126 11500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:28.795213 11500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "305f2cc43ed44c7c8dcf667974579e09"
format_stamp: "Formatted at 2026-09-29 19:09:28 on dist-test-slave-xz51"
server_key: "82635301b75a18b7f863af32a6c76052"
server_key_iv: "c09ff3e1cfec363ecf463a94a03c78cd"
server_key_version: "encryptionkey@0"
I20260929 19:09:28.795603 11500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "305f2cc43ed44c7c8dcf667974579e09"
format_stamp: "Formatted at 2026-09-29 19:09:28 on dist-test-slave-xz51"
server_key: "82635301b75a18b7f863af32a6c76052"
server_key_iv: "c09ff3e1cfec363ecf463a94a03c78cd"
server_key_version: "encryptionkey@0"
I20260929 19:09:28.796931 11500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:09:28.797672 11515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:28.797883 11500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:09:28.797988 11500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "305f2cc43ed44c7c8dcf667974579e09"
format_stamp: "Formatted at 2026-09-29 19:09:28 on dist-test-slave-xz51"
server_key: "82635301b75a18b7f863af32a6c76052"
server_key_iv: "c09ff3e1cfec363ecf463a94a03c78cd"
server_key_version: "encryptionkey@0"
I20260929 19:09:28.798072 11500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:28.807394 11500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:28.809160 11500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:28.809324 11500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:28.813246 11500 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:45953
I20260929 19:09:28.813315 11567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:45953 every 8 connection(s)
I20260929 19:09:28.813673 11500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:28.814554 11568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:28.816782 11568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09: Bootstrap starting.
I20260929 19:09:28.817366 11568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:28.817654 11568 log.cc:824] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:28.818416 11568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09: No bootstrap required, opened a new log
I20260929 19:09:28.820094 11568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305f2cc43ed44c7c8dcf667974579e09" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45953 } }
I20260929 19:09:28.820230 11568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:28.820251 11568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 305f2cc43ed44c7c8dcf667974579e09, State: Initialized, Role: FOLLOWER
I20260929 19:09:28.820430 11568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305f2cc43ed44c7c8dcf667974579e09" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45953 } }
I20260929 19:09:28.820497 11568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:28.820518 11568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:28.820585 11568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:28.821099 11568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305f2cc43ed44c7c8dcf667974579e09" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45953 } }
I20260929 19:09:28.821211 11568 leader_election.cc:302] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 305f2cc43ed44c7c8dcf667974579e09; no voters:
I20260929 19:09:28.821403 11568 leader_election.cc:288] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:28.821483 11571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:28.821653 11571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [term 1 LEADER]: Becoming Leader. State: Replica: 305f2cc43ed44c7c8dcf667974579e09, State: Running, Role: LEADER
I20260929 19:09:28.821684 11568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:28.821767 11571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305f2cc43ed44c7c8dcf667974579e09" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45953 } }
I20260929 19:09:28.821969 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 11500
I20260929 19:09:28.822075 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:28.822295 26939 external_mini_cluster.cc:1428] Setting key a849792b9d70329dd24985188ced4a78
I20260929 19:09:28.822830 11572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "305f2cc43ed44c7c8dcf667974579e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305f2cc43ed44c7c8dcf667974579e09" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45953 } } }
I20260929 19:09:28.822932 11572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:28.823380 11573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 305f2cc43ed44c7c8dcf667974579e09. Latest consensus state: current_term: 1 leader_uuid: "305f2cc43ed44c7c8dcf667974579e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305f2cc43ed44c7c8dcf667974579e09" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45953 } } }
I20260929 19:09:28.823664 11573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09 [sys.catalog]: This master's current role is: LEADER
W20260929 19:09:28.823858 11589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:09:28.823920 11589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:09:28.823976 11584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:28.824150 11584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:28.825539 11584 catalog_manager.cc:1347] Generated new cluster ID: 341d99f0256a422c979589dc82cd42e1
I20260929 19:09:28.825590 11584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:09:28 dist-test-slave-xz51 krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:28.833267 11584 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:28.834000 11584 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:28.835179 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:28.823000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.26.78.254:45953)
0929 19:09:28.823211 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:28.823221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:28.823253 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:09:28.823388 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:28.823395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:28.823539 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:28.823727 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:28.824473 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:28.825919 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:28.827019 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:28.827630 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:28.828062 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:28.830565 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:28.830570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:28.830570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:28.830584 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:28.833359 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:28.833851 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:28.833854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:28.833855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:28.833978 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:28.834213 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:28.834214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:28.834215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:28.834523 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:28.834683 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:28.834785 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 19:09:28.834952 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260929 19:09:28.840943 11584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09: Generated new TSK 0
I20260929 19:09:28.841095 11584 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:09:28.872951 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45953
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:28.954559 11594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:28.954818 11594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:28.954856 11594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:28.956003 11594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:28.956073 11594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:28.956117 11594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:09:28.957577 11594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:28.957909 11594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:28.958148 11594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:28.963116 11599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:28.963165 11600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:28.963165 11602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:28.963369 11594 server_base.cc:1034] running on GCE node
I20260929 19:09:28.963567 11594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:28.963816 11594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:28.964964 11594 hybrid_clock.cc:648] HybridClock initialized: now 1790708968964954 us; error 31 us; skew 500 ppm
Sep 29 19:09:28 dist-test-slave-xz51 krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708968, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:28.967449 11594 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:09:28.967829 11594 webserver.cc:460] Webserver started at http://127.26.78.193:45157/ using document root <none> and password file <none>
I20260929 19:09:28.968020 11594 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:28.968060 11594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:28.968147 11594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:28.969173 11594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "f10151008f4f460586ba5e08838f86f5"
format_stamp: "Formatted at 2026-09-29 19:09:28 on dist-test-slave-xz51"
server_key: "da4363a64bc613076af9ef302b48bb2a"
server_key_iv: "b5b34d62b8d342bc8cba0f7dd9d9efd1"
server_key_version: "encryptionkey@0"
I20260929 19:09:28.969475 11594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "f10151008f4f460586ba5e08838f86f5"
format_stamp: "Formatted at 2026-09-29 19:09:28 on dist-test-slave-xz51"
server_key: "da4363a64bc613076af9ef302b48bb2a"
server_key_iv: "b5b34d62b8d342bc8cba0f7dd9d9efd1"
server_key_version: "encryptionkey@0"
I20260929 19:09:28.970813 11594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:09:28.971506 11609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:28.971675 11594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:09:28.971779 11594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "f10151008f4f460586ba5e08838f86f5"
format_stamp: "Formatted at 2026-09-29 19:09:28 on dist-test-slave-xz51"
server_key: "da4363a64bc613076af9ef302b48bb2a"
server_key_iv: "b5b34d62b8d342bc8cba0f7dd9d9efd1"
server_key_version: "encryptionkey@0"
I20260929 19:09:28.971861 11594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:28.982522 11594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:28.984997 11594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:28.985170 11594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:28.985422 11594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:28.985756 11594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:28.985816 11594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:28.985944 11594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:28.985980 11594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:28.992023 11594 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:46117
I20260929 19:09:28.992069 11722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:46117 every 8 connection(s)
I20260929 19:09:28.992403 11594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:09:28.998303 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 11594
Sep 29 19:09:28 dist-test-slave-xz51 krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708968, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:28.998420 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:09:28.998603 26939 external_mini_cluster.cc:1428] Setting key f069498c61ec392d40d3c51a01629100
I20260929 19:09:29.002070 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:28.993153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:51579 (local address 127.26.78.254:45953)
0929 19:09:28.993226 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:28.993228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:28.993451 (+ 223us) server_negotiation.cc:407] Connection header received
0929 19:09:28.994017 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:28.994018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:28.994028 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:28.994089 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:28.995017 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:28.995502 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:28.996471 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:28.996652 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:28.996701 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:28.998828 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:28.998830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:28.998830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:28.998837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:29.000340 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.000932 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.000933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.000934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.001002 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.001248 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.001249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.001249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.001401 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:29.001469 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.001931 (+ 462us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.002006 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:09:29.002388 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:28.993132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45953 (local address 127.26.78.193:51579)
0929 19:09:28.993303 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:28.993356 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:28.993914 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:28.994145 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:28.994149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:28.994733 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:28.994936 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:28.994941 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:28.995581 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:28.995582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:28.996394 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:28.996396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:28.996725 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:28.996726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:28.996828 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:28.997307 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:28.997316 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:28.998711 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:29.000434 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:29.000442 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:29.000445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:29.000841 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:29.001077 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:29.001077 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:29.001078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:29.001184 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:29.001504 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:29.001512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:29.001733 (+ 221us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.001912 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.002094 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260929 19:09:29.002955 11723 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45953
I20260929 19:09:29.003050 11723 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:29.003240 11723 heartbeater.cc:511] Master 127.26.78.254:45953 requested a full tablet report, sending...
I20260929 19:09:29.003847 11532 ts_manager.cc:194] Registered new tserver with Master: f10151008f4f460586ba5e08838f86f5 (127.26.78.193:46117)
I20260929 19:09:29.004889 11532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:51579
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:09:29.025704 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45953
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:29.107231 11728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:29.107451 11728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:29.107476 11728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:29.108608 11728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:29.108644 11728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:29.108670 11728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:09:29.110275 11728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:29.110476 11728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:29.110661 11728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:29.115602 11734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:29.115720 11728 server_base.cc:1034] running on GCE node
W20260929 19:09:29.115695 11736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.115610 11733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:29.116076 11728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:29.116350 11728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:29.117522 11728 hybrid_clock.cc:648] HybridClock initialized: now 1790708969117497 us; error 35 us; skew 500 ppm
Sep 29 19:09:29 dist-test-slave-xz51 krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:29.120604 11728 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:09:29.121162 11728 webserver.cc:460] Webserver started at http://127.26.78.194:37219/ using document root <none> and password file <none>
I20260929 19:09:29.121510 11728 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:29.121585 11728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:29.121734 11728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:29.123292 11728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "2297d072163f4d9e8ffa8082c3889944"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "ca1816d7672c43ce51a13850ab84be1f"
server_key_iv: "abdb1c3984a6e5aac67f1f55a0e1f6a2"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.123663 11728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "2297d072163f4d9e8ffa8082c3889944"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "ca1816d7672c43ce51a13850ab84be1f"
server_key_iv: "abdb1c3984a6e5aac67f1f55a0e1f6a2"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.125554 11728 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260929 19:09:29.126750 11743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.126935 11728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:29.127034 11728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "2297d072163f4d9e8ffa8082c3889944"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "ca1816d7672c43ce51a13850ab84be1f"
server_key_iv: "abdb1c3984a6e5aac67f1f55a0e1f6a2"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.127120 11728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:29.153247 11728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:29.155799 11728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:29.155972 11728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:29.156241 11728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:29.156559 11728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:29.156611 11728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.156653 11728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:29.156694 11728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.162693 11728 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:45803
I20260929 19:09:29.162794 11856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:45803 every 8 connection(s)
I20260929 19:09:29.163090 11728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:09:29 dist-test-slave-xz51 krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:29.171028 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 11728
I20260929 19:09:29.171149 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:09:29.171376 26939 external_mini_cluster.cc:1428] Setting key e0323cfd4d0669e47b8b127a81ae9435
I20260929 19:09:29.173231 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.163895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:40487 (local address 127.26.78.254:45953)
0929 19:09:29.163961 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.163963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.164167 (+ 204us) server_negotiation.cc:407] Connection header received
0929 19:09:29.164721 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.164723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.164736 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.164789 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:29.165764 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.166309 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.167335 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.167572 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.167647 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.170011 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:29.170013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:29.170014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:29.170022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:29.171759 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.172167 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.172169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.172169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.172245 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.172505 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.172506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.172506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.172634 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:29.172709 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.173034 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.173131 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:09:29.173410 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.163857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45953 (local address 127.26.78.194:40487)
0929 19:09:29.164048 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.164086 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.164596 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.164869 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.164872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.165453 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:29.165657 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.165662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.166414 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.166416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.167240 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.167242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.167666 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.167667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.167786 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.168323 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:29.168332 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:29.169914 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:29.171845 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:29.171852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:29.171854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:29.172086 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:29.172313 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:29.172314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:29.172314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:29.172433 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:29.172720 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:29.172728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:29.172874 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.172989 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.173155 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260929 19:09:29.173923 11857 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45953
I20260929 19:09:29.174012 11857 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:29.174229 11857 heartbeater.cc:511] Master 127.26.78.254:45953 requested a full tablet report, sending...
I20260929 19:09:29.174664 11532 ts_manager.cc:194] Registered new tserver with Master: 2297d072163f4d9e8ffa8082c3889944 (127.26.78.194:45803)
I20260929 19:09:29.175189 11532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:40487
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:09:29.196907 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45953
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:29.278483 11862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:29.278698 11862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:29.278723 11862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:29.279834 11862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:29.279954 11862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:29.280009 11862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:09:29.281796 11862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:29.282042 11862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:29.282223 11862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:29.287384 11870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.287406 11867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.287405 11868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:29.287474 11862 server_base.cc:1034] running on GCE node
I20260929 19:09:29.287820 11862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:29.288066 11862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:29.289247 11862 hybrid_clock.cc:648] HybridClock initialized: now 1790708969289239 us; error 37 us; skew 500 ppm
Sep 29 19:09:29 dist-test-slave-xz51 krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:29.291986 11862 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:09:29.292436 11862 webserver.cc:460] Webserver started at http://127.26.78.195:38231/ using document root <none> and password file <none>
I20260929 19:09:29.292662 11862 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:29.292707 11862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:29.292928 11862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:29.294139 11862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "219dba672cf44cf48e3a028cbae11f1f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "2c6bb010726eec3cde5d6a34fa201bc3"
server_key_iv: "95bda74ba53bf20538677922e448bf4a"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.294510 11862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "219dba672cf44cf48e3a028cbae11f1f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "2c6bb010726eec3cde5d6a34fa201bc3"
server_key_iv: "95bda74ba53bf20538677922e448bf4a"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.295876 11862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:29.296785 11877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.297008 11862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:29.297120 11862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "219dba672cf44cf48e3a028cbae11f1f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "2c6bb010726eec3cde5d6a34fa201bc3"
server_key_iv: "95bda74ba53bf20538677922e448bf4a"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.297205 11862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:29.310803 11862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:29.312999 11862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:29.313159 11862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:29.313411 11862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:29.313755 11862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:29.313788 11862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.313812 11862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:29.313884 11862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.319622 11862 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:36359
I20260929 19:09:29.319720 11990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:36359 every 8 connection(s)
I20260929 19:09:29.320032 11862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:09:29.322196 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 11862
I20260929 19:09:29.322362 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:09:29.322589 26939 external_mini_cluster.cc:1428] Setting key 06419a3a5844c616f477401ed00a31e9
Sep 29 19:09:29 dist-test-slave-xz51 krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:29.330065 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.320864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:33321 (local address 127.26.78.254:45953)
0929 19:09:29.320963 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.320965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.321119 (+ 154us) server_negotiation.cc:407] Connection header received
0929 19:09:29.321729 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.321730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.321743 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.321797 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:29.323267 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.323880 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.324999 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.325216 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.325312 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.327410 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:29.327411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:29.327412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:29.327419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:29.328635 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.329063 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.329065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.329065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.329126 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.329379 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.329380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.329380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.329496 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:29.329573 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.329884 (+ 311us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.329967 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 19:09:29.330274 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.320777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45953 (local address 127.26.78.195:33321)
0929 19:09:29.320997 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.321034 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.321613 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.321901 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.321905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.322814 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:29.323135 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.323143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.323997 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.323998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.324904 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.324906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.325322 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.325323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.325453 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.325989 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:29.325997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:29.327307 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:29.328738 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:29.328751 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:29.328753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:29.328972 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:29.329193 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:29.329194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:29.329195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:29.329304 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:29.329561 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:29.329568 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:29.329710 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.329868 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.330035 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":116,"threads_started":1}
I20260929 19:09:29.330763 11991 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45953
I20260929 19:09:29.330870 11991 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:29.331072 11991 heartbeater.cc:511] Master 127.26.78.254:45953 requested a full tablet report, sending...
I20260929 19:09:29.331451 11532 ts_manager.cc:194] Registered new tserver with Master: 219dba672cf44cf48e3a028cbae11f1f (127.26.78.195:36359)
I20260929 19:09:29.331820 11532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:33321
I20260929 19:09:29.334249 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:09:29.342059 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.335698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.26.78.254:45953)
0929 19:09:29.335752 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.335754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.335873 (+ 119us) server_negotiation.cc:407] Connection header received
0929 19:09:29.335968 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.335969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.335979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.336031 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:29.336563 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.337045 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.337752 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.338030 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.338115 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.339318 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:29.339320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:29.339321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:29.339328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:29.340729 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.341179 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.341181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.341181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.341236 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:29.341471 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:29.341472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:29.341472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:29.341625 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:29.341733 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.341878 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.341957 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:09:29.343779 11532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:09:29.345453 11532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:09:29.352108 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.347898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:45803 (local address 127.0.0.1:39464)
0929 19:09:29.348041 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.348070 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.348136 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.348498 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.348501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.348504 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:29.348582 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.348584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.349636 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.349637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.350989 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.350992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.351769 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.351770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.351863 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.351877 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.351978 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.352032 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260929 19:09:29.352121 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.348117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.26.78.194:45803)
0929 19:09:29.348295 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.348302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.348313 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:09:29.348355 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.348357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.348374 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.348663 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:29.348739 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.349536 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.351081 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.351662 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.351765 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.351828 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.351997 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.352040 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":68,"threads_started":1}
I20260929 19:09:29.352339 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.347846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:36359 (local address 127.0.0.1:34046)
0929 19:09:29.348024 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.348070 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.348136 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.348637 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.348639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.348641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:29.348699 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.348717 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.350057 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.350059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.351098 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.351099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.352091 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.352092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.352178 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.352184 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.352233 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.352272 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":23,"thread_start_us":58,"threads_started":1}
I20260929 19:09:29.352543 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.348043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:46117 (local address 127.0.0.1:52132)
0929 19:09:29.348440 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.348511 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.348560 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.348825 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.348826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.348828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:29.348862 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.348864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.350206 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.350208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.351157 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.351160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.352092 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.352093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.352178 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.352186 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.352437 (+ 251us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.352488 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260929 19:09:29.352720 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.347933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.26.78.195:36359)
0929 19:09:29.348319 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.348323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.348331 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:09:29.348391 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.348393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.348409 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.348491 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:29.348802 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.349952 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.351188 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.351924 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.351998 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.352053 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.352516 (+ 463us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.352568 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":71,"threads_started":1}
I20260929 19:09:29.352787 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.348252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.26.78.193:46117)
0929 19:09:29.348435 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.348439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.348618 (+ 179us) server_negotiation.cc:407] Connection header received
0929 19:09:29.348662 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.348664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.348680 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.348747 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:29.348946 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.350098 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.351226 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.351927 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.351998 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.352052 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.352645 (+ 593us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.352694 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260929 19:09:29.353166 11791 tablet_service.cc:1467] Processing CreateTablet for tablet e9fe256fcddf49c5881bc8fe69df0470 (DEFAULT_TABLE table=test-table [id=b4bb57c46338453ba0d8ee6b68d549ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:29.353546 11791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fe256fcddf49c5881bc8fe69df0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:29.353507 11925 tablet_service.cc:1467] Processing CreateTablet for tablet e9fe256fcddf49c5881bc8fe69df0470 (DEFAULT_TABLE table=test-table [id=b4bb57c46338453ba0d8ee6b68d549ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:29.353513 11657 tablet_service.cc:1467] Processing CreateTablet for tablet e9fe256fcddf49c5881bc8fe69df0470 (DEFAULT_TABLE table=test-table [id=b4bb57c46338453ba0d8ee6b68d549ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:29.353855 11657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fe256fcddf49c5881bc8fe69df0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:29.353868 11925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fe256fcddf49c5881bc8fe69df0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:29.356496 12008 tablet_bootstrap.cc:492] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Bootstrap starting.
I20260929 19:09:29.356945 12009 tablet_bootstrap.cc:492] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Bootstrap starting.
I20260929 19:09:29.357237 12008 tablet_bootstrap.cc:654] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:29.357299 12010 tablet_bootstrap.cc:492] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: Bootstrap starting.
I20260929 19:09:29.357621 12009 tablet_bootstrap.cc:654] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:29.357918 12008 log.cc:824] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:29.357954 12010 tablet_bootstrap.cc:654] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:29.358217 12009 log.cc:824] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:29.358557 12010 log.cc:824] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:29.358811 12008 tablet_bootstrap.cc:492] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: No bootstrap required, opened a new log
I20260929 19:09:29.358870 12008 ts_tablet_manager.cc:1397] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:09:29.359274 12010 tablet_bootstrap.cc:492] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: No bootstrap required, opened a new log
I20260929 19:09:29.359347 12010 ts_tablet_manager.cc:1397] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:09:29.360551 12008 raft_consensus.cc:348] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.360721 12008 raft_consensus.cc:374] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:29.360767 12008 raft_consensus.cc:729] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219dba672cf44cf48e3a028cbae11f1f, State: Initialized, Role: FOLLOWER
I20260929 19:09:29.360736 12010 raft_consensus.cc:348] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.360893 12008 consensus_queue.cc:260] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.360941 12010 raft_consensus.cc:374] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:29.361003 12010 raft_consensus.cc:729] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10151008f4f460586ba5e08838f86f5, State: Initialized, Role: FOLLOWER
I20260929 19:09:29.361112 12010 consensus_queue.cc:260] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.361167 12008 ts_tablet_manager.cc:1428] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:09:29.361291 11991 heartbeater.cc:503] Master 127.26.78.254:45953 was elected leader, sending a full tablet report...
I20260929 19:09:29.361379 12010 ts_tablet_manager.cc:1428] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:09:29.361503 11723 heartbeater.cc:503] Master 127.26.78.254:45953 was elected leader, sending a full tablet report...
I20260929 19:09:29.361608 12009 tablet_bootstrap.cc:492] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: No bootstrap required, opened a new log
I20260929 19:09:29.361667 12009 ts_tablet_manager.cc:1397] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 19:09:29.363309 12009 raft_consensus.cc:348] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.363473 12009 raft_consensus.cc:374] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:29.363516 12009 raft_consensus.cc:729] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2297d072163f4d9e8ffa8082c3889944, State: Initialized, Role: FOLLOWER
I20260929 19:09:29.363631 12009 consensus_queue.cc:260] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.363899 12009 ts_tablet_manager.cc:1428] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:09:29.363967 11857 heartbeater.cc:503] Master 127.26.78.254:45953 was elected leader, sending a full tablet report...
W20260929 19:09:29.413789 11858 tablet.cc:2369] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:09:29.478817 12015 raft_consensus.cc:482] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:29.478986 12015 raft_consensus.cc:504] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.479403 12015 leader_election.cc:288] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 219dba672cf44cf48e3a028cbae11f1f (127.26.78.195:36359), 2297d072163f4d9e8ffa8082c3889944 (127.26.78.194:45803)
I20260929 19:09:29.482402 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.479670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:33411 (local address 127.26.78.195:36359)
0929 19:09:29.479718 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.479720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.479731 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:09:29.479794 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.479796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.479806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.479889 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:29.480069 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.480577 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.481554 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.482046 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.482114 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.482123 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.482294 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.482334 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:09:29.482441 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.479606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:36359 (local address 127.26.78.193:33411)
0929 19:09:29.479668 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.479681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.479727 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.479910 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.479911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.479912 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:29.479990 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.479992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.480661 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.480662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.481454 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.481456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.482117 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.482118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.482207 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.482211 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.482276 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.482319 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 19:09:29.482463 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.479704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:60477 (local address 127.26.78.194:45803)
0929 19:09:29.479753 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.479755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.479881 (+ 126us) server_negotiation.cc:407] Connection header received
0929 19:09:29.479910 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.479911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.479920 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.479980 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:29.480169 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.480641 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.481591 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.482099 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.482177 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.482186 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.482340 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.482389 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:09:29.482596 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.479624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:45803 (local address 127.26.78.193:60477)
0929 19:09:29.479765 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.479774 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.479839 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.480000 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.480001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.480002 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:29.480098 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.480100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.480729 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.480730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.481463 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.481465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.482178 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.482178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.482254 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.482262 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.482334 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.482374 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260929 19:09:29.482802 11945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fe256fcddf49c5881bc8fe69df0470" candidate_uuid: "f10151008f4f460586ba5e08838f86f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "219dba672cf44cf48e3a028cbae11f1f" is_pre_election: true
I20260929 19:09:29.482959 11945 raft_consensus.cc:2454] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10151008f4f460586ba5e08838f86f5 in term 0.
I20260929 19:09:29.482939 11811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fe256fcddf49c5881bc8fe69df0470" candidate_uuid: "f10151008f4f460586ba5e08838f86f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2297d072163f4d9e8ffa8082c3889944" is_pre_election: true
I20260929 19:09:29.483086 11811 raft_consensus.cc:2454] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10151008f4f460586ba5e08838f86f5 in term 0.
I20260929 19:09:29.483203 11612 leader_election.cc:302] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 219dba672cf44cf48e3a028cbae11f1f, f10151008f4f460586ba5e08838f86f5; no voters:
I20260929 19:09:29.483343 12015 raft_consensus.cc:2781] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:09:29.483410 12015 raft_consensus.cc:482] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:09:29.483429 12015 raft_consensus.cc:3037] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:29.484241 12015 raft_consensus.cc:504] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.484390 12015 leader_election.cc:288] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [CANDIDATE]: Term 1 election: Requested vote from peers 219dba672cf44cf48e3a028cbae11f1f (127.26.78.195:36359), 2297d072163f4d9e8ffa8082c3889944 (127.26.78.194:45803)
I20260929 19:09:29.484524 11945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fe256fcddf49c5881bc8fe69df0470" candidate_uuid: "f10151008f4f460586ba5e08838f86f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "219dba672cf44cf48e3a028cbae11f1f"
I20260929 19:09:29.484565 11811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fe256fcddf49c5881bc8fe69df0470" candidate_uuid: "f10151008f4f460586ba5e08838f86f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2297d072163f4d9e8ffa8082c3889944"
I20260929 19:09:29.484608 11945 raft_consensus.cc:3037] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:29.484652 11811 raft_consensus.cc:3037] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:29.485357 11811 raft_consensus.cc:2454] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10151008f4f460586ba5e08838f86f5 in term 1.
I20260929 19:09:29.485387 11945 raft_consensus.cc:2454] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10151008f4f460586ba5e08838f86f5 in term 1.
I20260929 19:09:29.485582 11610 leader_election.cc:302] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2297d072163f4d9e8ffa8082c3889944, f10151008f4f460586ba5e08838f86f5; no voters:
I20260929 19:09:29.485677 12015 raft_consensus.cc:2781] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:29.485878 12015 raft_consensus.cc:686] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 1 LEADER]: Becoming Leader. State: Replica: f10151008f4f460586ba5e08838f86f5, State: Running, Role: LEADER
I20260929 19:09:29.486037 12015 consensus_queue.cc:237] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } }
I20260929 19:09:29.487026 11532 catalog_manager.cc:5495] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f10151008f4f460586ba5e08838f86f5 (127.26.78.193). New cstate: current_term: 1 leader_uuid: "f10151008f4f460586ba5e08838f86f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f10151008f4f460586ba5e08838f86f5" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 46117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 } health_report { overall_health: UNKNOWN } } }
W20260929 19:09:29.493119 11724 tablet.cc:2369] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:09:29.495873 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.493756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.26.78.193:46117)
0929 19:09:29.493792 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.493794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.493805 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:09:29.493934 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.493935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.493950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.494011 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:09:29.494107 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.494562 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.495269 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.495452 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.495500 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.495632 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:09:29.495719 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:09:29.495776 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.495811 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.495826 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:09:29.497516 11811 raft_consensus.cc:1261] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 1 FOLLOWER]: Refusing update from remote peer f10151008f4f460586ba5e08838f86f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:29.497512 11945 raft_consensus.cc:1261] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 1 FOLLOWER]: Refusing update from remote peer f10151008f4f460586ba5e08838f86f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:29.497860 12015 consensus_queue.cc:1035] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2297d072163f4d9e8ffa8082c3889944" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 45803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:29.497979 12018 consensus_queue.cc:1035] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "219dba672cf44cf48e3a028cbae11f1f" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 36359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:29.502461 11531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:29.502554 11531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:29.503327 11531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 305f2cc43ed44c7c8dcf667974579e09: Sending DeleteTablet for 3 replicas of tablet e9fe256fcddf49c5881bc8fe69df0470
I20260929 19:09:29.503680 11657 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fe256fcddf49c5881bc8fe69df0470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:29 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:52132
I20260929 19:09:29.503712 11791 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fe256fcddf49c5881bc8fe69df0470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:29 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:39464
I20260929 19:09:29.503947 12031 tablet_replica.cc:330] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5: stopping tablet replica
I20260929 19:09:29.503948 12032 tablet_replica.cc:330] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: stopping tablet replica
I20260929 19:09:29.503949 11925 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fe256fcddf49c5881bc8fe69df0470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:29 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:34046
I20260929 19:09:29.504046 12032 raft_consensus.cc:2229] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:29.504139 12031 raft_consensus.cc:2229] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:09:29.504154 12032 raft_consensus.cc:2258] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:29.504372 12031 raft_consensus.cc:2258] T e9fe256fcddf49c5881bc8fe69df0470 P f10151008f4f460586ba5e08838f86f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:29.504513 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 11594
I20260929 19:09:29.505084 12032 ts_tablet_manager.cc:1905] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:29.506839 12032 ts_tablet_manager.cc:1918] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:29.506910 12032 log.cc:1196] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/e9fe256fcddf49c5881bc8fe69df0470
I20260929 19:09:29.507189 12032 ts_tablet_manager.cc:1939] T e9fe256fcddf49c5881bc8fe69df0470 P 2297d072163f4d9e8ffa8082c3889944: Deleting consensus metadata
I20260929 19:09:29.507611 11516 catalog_manager.cc:4841] TS 2297d072163f4d9e8ffa8082c3889944 (127.26.78.194:45803): tablet e9fe256fcddf49c5881bc8fe69df0470 (table test-table [id=b4bb57c46338453ba0d8ee6b68d549ac]) successfully deleted
W20260929 19:09:29.510391 11518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:46117 (error 108)
I20260929 19:09:29.510603 12033 tablet_replica.cc:330] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: stopping tablet replica
I20260929 19:09:29.510720 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 11728
I20260929 19:09:29.510736 12033 raft_consensus.cc:2229] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:29.510848 12033 raft_consensus.cc:2258] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:29.511687 12033 ts_tablet_manager.cc:1905] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:29.513013 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.512896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:46117 (local address 127.0.0.1:52148)
0929 19:09:29.512939 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.512983 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:46117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260929 19:09:29.513163 11518 catalog_manager.cc:4584] TS f10151008f4f460586ba5e08838f86f5 (127.26.78.193:46117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9fe256fcddf49c5881bc8fe69df0470: Network error: Client connection negotiation failed: client connection to 127.26.78.193:46117: connect: Connection refused (error 111)
I20260929 19:09:29.513286 12033 ts_tablet_manager.cc:1918] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:29.513371 12033 log.cc:1196] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/e9fe256fcddf49c5881bc8fe69df0470
I20260929 19:09:29.513619 12033 ts_tablet_manager.cc:1939] T e9fe256fcddf49c5881bc8fe69df0470 P 219dba672cf44cf48e3a028cbae11f1f: Deleting consensus metadata
I20260929 19:09:29.514159 11518 catalog_manager.cc:4841] TS 219dba672cf44cf48e3a028cbae11f1f (127.26.78.195:36359): tablet e9fe256fcddf49c5881bc8fe69df0470 (table test-table [id=b4bb57c46338453ba0d8ee6b68d549ac]) successfully deleted
I20260929 19:09:29.516340 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 11862
I20260929 19:09:29.521117 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 11500
2026-09-29T19:09:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1995 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T19:09:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:29Z Disabled control of system clock
I20260929 19:09:29.549182 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38355
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:38355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:09:29.632644 12038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:29.632885 12038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:29.632907 12038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:29.634029 12038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:29.634068 12038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:29.634081 12038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:29.634094 12038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:29.634106 12038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:29.635404 12038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:38355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:29.635605 12038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:29.635866 12038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:29.641053 12046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.641073 12044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:29.641160 12038 server_base.cc:1034] running on GCE node
W20260929 19:09:29.641073 12043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:29.641580 12038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:29.641995 12038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:29.643168 12038 hybrid_clock.cc:648] HybridClock initialized: now 1790708969643153 us; error 27 us; skew 500 ppm
I20260929 19:09:29.644469 12038 webserver.cc:460] Webserver started at http://127.26.78.254:37007/ using document root <none> and password file <none>
I20260929 19:09:29.644747 12038 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:29.644819 12038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:29.644989 12038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:29.646190 12038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "eb86453008904a11abbfebe4bd6fcf1f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "33187238a967173b0674dcc3d3c2c739"
server_key_iv: "2e6be75aa30fefb4533cf2ce1cb492ed"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.646555 12038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "eb86453008904a11abbfebe4bd6fcf1f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "33187238a967173b0674dcc3d3c2c739"
server_key_iv: "2e6be75aa30fefb4533cf2ce1cb492ed"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.647867 12038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:29.648609 12052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.648780 12038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:29.648904 12038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "eb86453008904a11abbfebe4bd6fcf1f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "33187238a967173b0674dcc3d3c2c739"
server_key_iv: "2e6be75aa30fefb4533cf2ce1cb492ed"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.648983 12038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:29.670418 12038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:29.670792 12038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:29.670970 12038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:29.675170 12038 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:38355
I20260929 19:09:29.675209 12104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:38355 every 8 connection(s)
I20260929 19:09:29.675570 12038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:29.676533 12105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:29.678850 12105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f: Bootstrap starting.
I20260929 19:09:29.679448 12105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:29.679767 12105 log.cc:824] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:29.680474 12105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f: No bootstrap required, opened a new log
I20260929 19:09:29.682168 12105 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb86453008904a11abbfebe4bd6fcf1f" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38355 } }
I20260929 19:09:29.682307 12105 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:29.682330 12105 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb86453008904a11abbfebe4bd6fcf1f, State: Initialized, Role: FOLLOWER
I20260929 19:09:29.682510 12105 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb86453008904a11abbfebe4bd6fcf1f" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38355 } }
I20260929 19:09:29.682597 12105 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:29.682639 12105 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:29.682688 12105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:29.683295 12105 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb86453008904a11abbfebe4bd6fcf1f" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38355 } }
I20260929 19:09:29.683415 12105 leader_election.cc:302] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb86453008904a11abbfebe4bd6fcf1f; no voters:
I20260929 19:09:29.683604 12105 leader_election.cc:288] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:29.683871 12105 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:29.683933 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12038
I20260929 19:09:29.684020 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:29.684216 26939 external_mini_cluster.cc:1428] Setting key 19325812834d3d112c5ef6e9f9e8ed13
I20260929 19:09:29.684428 12108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:29.684576 12108 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [term 1 LEADER]: Becoming Leader. State: Replica: eb86453008904a11abbfebe4bd6fcf1f, State: Running, Role: LEADER
I20260929 19:09:29.684682 12108 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb86453008904a11abbfebe4bd6fcf1f" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38355 } }
I20260929 19:09:29.685097 12109 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb86453008904a11abbfebe4bd6fcf1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb86453008904a11abbfebe4bd6fcf1f" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38355 } } }
I20260929 19:09:29.685178 12109 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:29.685343 12110 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb86453008904a11abbfebe4bd6fcf1f. Latest consensus state: current_term: 1 leader_uuid: "eb86453008904a11abbfebe4bd6fcf1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb86453008904a11abbfebe4bd6fcf1f" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38355 } } }
I20260929 19:09:29.685407 12110 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:29.685587 12117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:29.686113 12117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:29.688803 12117 catalog_manager.cc:1347] Generated new cluster ID: 524c1e84b4ae46ab93ece60c08a986a2
I20260929 19:09:29.688875 12117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:29.690531 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.684709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.26.78.254:38355)
0929 19:09:29.685074 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.685085 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.685129 (+ 44us) server_negotiation.cc:407] Connection header received
0929 19:09:29.685268 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.685277 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.685377 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.685784 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:29.685941 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.687322 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.688420 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.689018 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.689152 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.689873 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:29.689877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:29.689877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:29.690059 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:29.690092 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:29.690103 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:29.690176 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.690259 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.690340 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":61,"threads_started":1}
I20260929 19:09:29.714085 12117 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:29.714636 12117 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:29.727423 12117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f: Generated new TSK 0
I20260929 19:09:29.727662 12117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:29.733652 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38355
--builtin_ntp_servers=127.26.78.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:09:29.817518 12129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:29.817773 12129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:29.817811 12129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:29.819080 12129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:29.819150 12129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:29.819195 12129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:09:29.820600 12129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:29.820875 12129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:29.821178 12129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:29.826407 12134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.826426 12137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.826524 12135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:29.826701 12129 server_base.cc:1034] running on GCE node
I20260929 19:09:29.826920 12129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:29.827162 12129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:29.828320 12129 hybrid_clock.cc:648] HybridClock initialized: now 1790708969828297 us; error 35 us; skew 500 ppm
I20260929 19:09:29.829639 12129 webserver.cc:460] Webserver started at http://127.26.78.193:44429/ using document root <none> and password file <none>
I20260929 19:09:29.829926 12129 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:29.829974 12129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:29.830129 12129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:29.831293 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "caedd4d055c5473086ac03fc296c9b2f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "70e2f3d66f57c768ee7f5adfc866fe95"
server_key_iv: "7cedd8dfe723bf176d21e16ef104f45f"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.831676 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "caedd4d055c5473086ac03fc296c9b2f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "70e2f3d66f57c768ee7f5adfc866fe95"
server_key_iv: "7cedd8dfe723bf176d21e16ef104f45f"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.832983 12129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:29.833786 12143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.834031 12129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:29.834103 12129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "caedd4d055c5473086ac03fc296c9b2f"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "70e2f3d66f57c768ee7f5adfc866fe95"
server_key_iv: "7cedd8dfe723bf176d21e16ef104f45f"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.834190 12129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:29.850917 12129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:29.851246 12129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:29.851419 12129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:29.851668 12129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:29.852046 12129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:29.852079 12129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.852133 12129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:29.852159 12129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.857906 12129 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:43329
I20260929 19:09:29.857965 12256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:43329 every 8 connection(s)
I20260929 19:09:29.858328 12129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:09:29.858837 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12129
I20260929 19:09:29.858937 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:09:29.859177 26939 external_mini_cluster.cc:1428] Setting key 5ac8d9fc457ded42c45570f5e24cd4bf
I20260929 19:09:29.860208 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38355
--builtin_ntp_servers=127.26.78.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:09:29.864127 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.859104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:44385 (local address 127.26.78.254:38355)
0929 19:09:29.859189 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.859191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.859493 (+ 302us) server_negotiation.cc:407] Connection header received
0929 19:09:29.860161 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.860163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.860177 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.860230 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:29.861013 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.861567 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.862646 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.862877 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.862952 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.863836 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:29.863838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:29.863838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:29.863843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:29.863864 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:29.863869 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:29.863921 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.864012 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.864045 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:09:29.864382 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.859101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38355 (local address 127.26.78.193:44385)
0929 19:09:29.859332 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.859389 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.860053 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.860312 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.860317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.860328 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:29.860897 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.860901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.861671 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.861672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.862540 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.862542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.862971 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.862972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.863116 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.863677 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:29.863678 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:29.863707 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:29.863730 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:29.863931 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:29.863936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:29.863939 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.864009 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.864113 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
I20260929 19:09:29.864822 12257 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38355
I20260929 19:09:29.864914 12257 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:29.865120 12257 heartbeater.cc:511] Master 127.26.78.254:38355 requested a full tablet report, sending...
I20260929 19:09:29.865620 12069 ts_manager.cc:194] Registered new tserver with Master: caedd4d055c5473086ac03fc296c9b2f (127.26.78.193:43329)
I20260929 19:09:29.866274 12069 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.193:44385
W20260929 19:09:29.941978 12260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:29.942232 12260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:29.942271 12260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:29.943420 12260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:29.943490 12260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:29.943531 12260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:09:29.944936 12260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:29.945211 12260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:29.945511 12260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:29.950626 12268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.950655 12265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:29.950619 12266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:29.950728 12260 server_base.cc:1034] running on GCE node
I20260929 19:09:29.951090 12260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:29.951357 12260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:29.952553 12260 hybrid_clock.cc:648] HybridClock initialized: now 1790708969952545 us; error 37 us; skew 500 ppm
I20260929 19:09:29.953943 12260 webserver.cc:460] Webserver started at http://127.26.78.194:42783/ using document root <none> and password file <none>
I20260929 19:09:29.954210 12260 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:29.954283 12260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:29.954437 12260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:29.955516 12260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "d55914dc36104b5b907f72368538e755"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "d1a47273e88db57bddbb6423670a7eb3"
server_key_iv: "bb9a3451ae43d7a8535fff719f1899eb"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.955886 12260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "d55914dc36104b5b907f72368538e755"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "d1a47273e88db57bddbb6423670a7eb3"
server_key_iv: "bb9a3451ae43d7a8535fff719f1899eb"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.957280 12260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:29.958089 12274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.958289 12260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:29.958397 12260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d55914dc36104b5b907f72368538e755"
format_stamp: "Formatted at 2026-09-29 19:09:29 on dist-test-slave-xz51"
server_key: "d1a47273e88db57bddbb6423670a7eb3"
server_key_iv: "bb9a3451ae43d7a8535fff719f1899eb"
server_key_version: "encryptionkey@0"
I20260929 19:09:29.958472 12260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:29.967115 12260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:29.967445 12260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:29.967602 12260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:29.967839 12260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:29.968175 12260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:29.968230 12260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.968271 12260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:29.968295 12260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:29.973999 12260 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:36881
I20260929 19:09:29.974037 12387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:36881 every 8 connection(s)
I20260929 19:09:29.974392 12260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:09:29.975052 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12260
I20260929 19:09:29.975172 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:09:29.975419 26939 external_mini_cluster.cc:1428] Setting key fb8e5859c2a79f51f7914e094d205499
I20260929 19:09:29.976445 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38355
--builtin_ntp_servers=127.26.78.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:09:29.980852 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.975190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:53861 (local address 127.26.78.254:38355)
0929 19:09:29.975272 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:29.975275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:29.975538 (+ 263us) server_negotiation.cc:407] Connection header received
0929 19:09:29.976122 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:29.976123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:29.976157 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:29.976222 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:29.976970 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.977752 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.979242 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.979591 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.979667 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.980509 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:29.980510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:29.980511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:29.980517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:29.980537 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:29.980542 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:29.980612 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:29.980698 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 19:09:29.980731 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:09:29.980954 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:29.975198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38355 (local address 127.26.78.194:53861)
0929 19:09:29.975396 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:29.975437 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:29.976003 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:29.976287 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:29.976290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:29.976298 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:29.976873 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.976879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.977891 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.977893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.979150 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:29.979153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:29.979678 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:29.979679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:29.979787 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:29.980367 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:29.980367 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:29.980397 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:29.980419 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:29.980616 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:29.980621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:29.980624 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:29.980677 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:09:29.980757 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260929 19:09:29.981544 12388 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38355
I20260929 19:09:29.981627 12388 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:29.981880 12388 heartbeater.cc:511] Master 127.26.78.254:38355 requested a full tablet report, sending...
I20260929 19:09:29.982313 12069 ts_manager.cc:194] Registered new tserver with Master: d55914dc36104b5b907f72368538e755 (127.26.78.194:36881)
I20260929 19:09:29.982869 12069 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.194:53861
W20260929 19:09:30.059823 12391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:30.060058 12391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:30.060082 12391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:30.061339 12391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:30.061386 12391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:30.061414 12391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:09:30.063051 12391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:38355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:30.063325 12391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:30.063642 12391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:30.068873 12396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:30.068926 12397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:30.069001 12391 server_base.cc:1034] running on GCE node
W20260929 19:09:30.068904 12399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:30.069295 12391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:30.069550 12391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:30.070712 12391 hybrid_clock.cc:648] HybridClock initialized: now 1790708970070705 us; error 30 us; skew 500 ppm
I20260929 19:09:30.072135 12391 webserver.cc:460] Webserver started at http://127.26.78.195:46839/ using document root <none> and password file <none>
I20260929 19:09:30.072393 12391 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:30.072443 12391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:30.072604 12391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:30.073701 12391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "b25c4bfb8725432e875e1134ba8841c8"
format_stamp: "Formatted at 2026-09-29 19:09:30 on dist-test-slave-xz51"
server_key: "ee1e98b1232054d49ed74194d1bf31a2"
server_key_iv: "a267b05748494bd80b0bf2e2decbf86e"
server_key_version: "encryptionkey@0"
I20260929 19:09:30.074146 12391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "b25c4bfb8725432e875e1134ba8841c8"
format_stamp: "Formatted at 2026-09-29 19:09:30 on dist-test-slave-xz51"
server_key: "ee1e98b1232054d49ed74194d1bf31a2"
server_key_iv: "a267b05748494bd80b0bf2e2decbf86e"
server_key_version: "encryptionkey@0"
I20260929 19:09:30.075482 12391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:30.076254 12405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:30.076429 12391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:30.076522 12391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "b25c4bfb8725432e875e1134ba8841c8"
format_stamp: "Formatted at 2026-09-29 19:09:30 on dist-test-slave-xz51"
server_key: "ee1e98b1232054d49ed74194d1bf31a2"
server_key_iv: "a267b05748494bd80b0bf2e2decbf86e"
server_key_version: "encryptionkey@0"
I20260929 19:09:30.076579 12391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:30.111083 12391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:30.111392 12391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:30.111514 12391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:30.111703 12391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:30.112088 12391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:30.112125 12391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:30.112149 12391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:30.112197 12391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:30.117865 12391 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:35927
I20260929 19:09:30.117960 12518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:35927 every 8 connection(s)
I20260929 19:09:30.118333 12391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:09:30.122457 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12391
I20260929 19:09:30.122558 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:09:30.122802 26939 external_mini_cluster.cc:1428] Setting key c434b29b090a7efeb4fd6bbefb951b88
I20260929 19:09:30.126570 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.119162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:49641 (local address 127.26.78.254:38355)
0929 19:09:30.119226 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.119229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.119484 (+ 255us) server_negotiation.cc:407] Connection header received
0929 19:09:30.120022 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.120024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.120034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.120096 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:30.120750 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.121303 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.122839 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.123187 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.123262 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.126178 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:30.126180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:30.126180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:30.126186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:30.126225 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:30.126231 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:30.126281 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.126427 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.126478 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:09:30.126787 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.119156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:38355 (local address 127.26.78.195:49641)
0929 19:09:30.119346 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:30.119389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:30.119928 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:30.120151 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:30.120155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:30.120163 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:30.120673 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.120678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.121399 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.121400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.122746 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.122749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.125085 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.125086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.125287 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.125975 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:30.125976 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:30.126020 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:30.126051 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:30.126303 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:30.126309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:30.126313 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:30.126403 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:09:30.126523 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260929 19:09:30.127285 12519 heartbeater.cc:347] Connected to a master server at 127.26.78.254:38355
I20260929 19:09:30.127417 12519 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:30.127645 12519 heartbeater.cc:511] Master 127.26.78.254:38355 requested a full tablet report, sending...
I20260929 19:09:30.128085 12069 ts_manager.cc:194] Registered new tserver with Master: b25c4bfb8725432e875e1134ba8841c8 (127.26.78.195:35927)
I20260929 19:09:30.128494 12069 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.195:49641
I20260929 19:09:30.134533 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:09:30.138708 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.136031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.26.78.254:38355)
0929 19:09:30.136109 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.136111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.136226 (+ 115us) server_negotiation.cc:407] Connection header received
0929 19:09:30.136269 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.136269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.136281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.136330 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:30.136451 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.136921 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.137699 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.137928 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.137977 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.138526 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:30.138528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:30.138528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:30.138532 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:30.138545 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:30.138549 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:30.138595 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.138630 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.138651 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 19:09:30.140110 12069 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:09:30.141701 12069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:09:30.148355 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.143920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43329 (local address 127.0.0.1:43736)
0929 19:09:30.144597 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:30.144606 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:30.144647 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:30.144966 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:30.144967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:30.144969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:30.145017 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.145018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.145973 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.145974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.147289 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.147291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.148060 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.148061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.148131 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.148144 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 19:09:30.148222 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 19:09:30.148276 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":94,"threads_started":1}
I20260929 19:09:30.148372 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.144146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.26.78.193:43329)
0929 19:09:30.144432 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.144436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.144701 (+ 265us) server_negotiation.cc:407] Connection header received
0929 19:09:30.144792 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.144795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.144811 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.144898 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:30.145075 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.145885 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.147391 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.147972 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.148062 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.148158 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.148236 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.148273 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":75,"threads_started":1}
I20260929 19:09:30.148738 12532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.144152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.26.78.195:35927)
0929 19:09:30.144344 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.144349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.144456 (+ 107us) server_negotiation.cc:407] Connection header received
0929 19:09:30.144842 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.144845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.144861 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.144953 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:30.145144 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.146214 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.147344 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.148028 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.148098 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.148194 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.148606 (+ 412us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.148648 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260929 19:09:30.148488 12528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.143904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:35927 (local address 127.0.0.1:39616)
0929 19:09:30.144365 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:30.144375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:30.144487 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:30.145026 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:30.145027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:30.145029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:30.145067 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.145069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.146289 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.146290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.147239 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.147242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.148234 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.148235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.148310 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.148314 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:30.148371 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 19:09:30.148410 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260929 19:09:30.149000 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.144086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.26.78.194:36881)
0929 19:09:30.144275 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.144279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.144422 (+ 143us) server_negotiation.cc:407] Connection header received
0929 19:09:30.144518 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.144521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.144540 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.144628 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:30.144859 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.146121 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.147588 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.148315 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.148379 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.148865 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.148892 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.148928 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260929 19:09:30.149183 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.143904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:36881 (local address 127.0.0.1:50852)
0929 19:09:30.144290 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:30.144333 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:30.144422 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:30.144681 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:30.144685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:30.144688 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:30.144765 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.144767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.146205 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.146206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.147509 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.147511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.148665 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.148666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.148724 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.148728 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:30.148778 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 19:09:30.148819 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":67,"threads_started":1}
I20260929 19:09:30.149353 12191 tablet_service.cc:1467] Processing CreateTablet for tablet baa4c0dcd9554497a8066f6604ca4276 (DEFAULT_TABLE table=test-table [id=d31131263c7d4cfa947745aca2bd0540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:30.149760 12191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa4c0dcd9554497a8066f6604ca4276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:30.150010 12322 tablet_service.cc:1467] Processing CreateTablet for tablet baa4c0dcd9554497a8066f6604ca4276 (DEFAULT_TABLE table=test-table [id=d31131263c7d4cfa947745aca2bd0540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:30.150359 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa4c0dcd9554497a8066f6604ca4276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:30.150532 12453 tablet_service.cc:1467] Processing CreateTablet for tablet baa4c0dcd9554497a8066f6604ca4276 (DEFAULT_TABLE table=test-table [id=d31131263c7d4cfa947745aca2bd0540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:30.150858 12453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa4c0dcd9554497a8066f6604ca4276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:30.153404 12534 tablet_bootstrap.cc:492] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Bootstrap starting.
I20260929 19:09:30.153637 12533 tablet_bootstrap.cc:492] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f: Bootstrap starting.
I20260929 19:09:30.154448 12535 tablet_bootstrap.cc:492] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Bootstrap starting.
I20260929 19:09:30.154614 12534 tablet_bootstrap.cc:654] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:30.154920 12534 log.cc:824] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:30.155130 12535 tablet_bootstrap.cc:654] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:30.155416 12535 log.cc:824] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:30.155542 12533 tablet_bootstrap.cc:654] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:30.155864 12533 log.cc:824] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:30.156209 12535 tablet_bootstrap.cc:492] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: No bootstrap required, opened a new log
I20260929 19:09:30.156291 12535 ts_tablet_manager.cc:1397] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:09:30.156435 12534 tablet_bootstrap.cc:492] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: No bootstrap required, opened a new log
I20260929 19:09:30.156490 12534 ts_tablet_manager.cc:1397] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 19:09:30.158442 12535 raft_consensus.cc:348] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.158583 12535 raft_consensus.cc:374] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:30.158607 12535 raft_consensus.cc:729] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b25c4bfb8725432e875e1134ba8841c8, State: Initialized, Role: FOLLOWER
I20260929 19:09:30.158514 12534 raft_consensus.cc:348] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.158681 12534 raft_consensus.cc:374] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:30.158740 12534 raft_consensus.cc:729] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55914dc36104b5b907f72368538e755, State: Initialized, Role: FOLLOWER
I20260929 19:09:30.158731 12535 consensus_queue.cc:260] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.158859 12534 consensus_queue.cc:260] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.159026 12519 heartbeater.cc:503] Master 127.26.78.254:38355 was elected leader, sending a full tablet report...
I20260929 19:09:30.159116 12534 ts_tablet_manager.cc:1428] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:30.159139 12388 heartbeater.cc:503] Master 127.26.78.254:38355 was elected leader, sending a full tablet report...
I20260929 19:09:30.159011 12535 ts_tablet_manager.cc:1428] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:30.159557 12533 tablet_bootstrap.cc:492] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f: No bootstrap required, opened a new log
I20260929 19:09:30.159605 12533 ts_tablet_manager.cc:1397] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260929 19:09:30.161757 12533 raft_consensus.cc:348] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.162009 12533 raft_consensus.cc:374] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:30.162062 12533 raft_consensus.cc:729] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caedd4d055c5473086ac03fc296c9b2f, State: Initialized, Role: FOLLOWER
I20260929 19:09:30.162185 12533 consensus_queue.cc:260] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.162448 12533 ts_tablet_manager.cc:1428] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:30.162509 12257 heartbeater.cc:503] Master 127.26.78.254:38355 was elected leader, sending a full tablet report...
I20260929 19:09:30.165584 12540 raft_consensus.cc:482] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:30.165661 12540 raft_consensus.cc:504] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.166028 12540 leader_election.cc:288] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b25c4bfb8725432e875e1134ba8841c8 (127.26.78.195:35927), caedd4d055c5473086ac03fc296c9b2f (127.26.78.193:43329)
I20260929 19:09:30.169898 12532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.166276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:40637 (local address 127.26.78.195:35927)
0929 19:09:30.166408 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.166410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.166499 (+ 89us) server_negotiation.cc:407] Connection header received
0929 19:09:30.166542 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.166543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.166551 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.166607 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:30.166778 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.167444 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.168927 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.169480 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.169560 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.169567 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.169757 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.169805 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260929 19:09:30.169894 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.166247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:45485 (local address 127.26.78.193:43329)
0929 19:09:30.166291 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.166293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.166306 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:09:30.166378 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.166379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.166390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.166446 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:30.166595 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.167406 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.168927 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.169483 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.169560 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.169567 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.169757 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.169805 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:09:30.169946 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.166197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43329 (local address 127.26.78.194:45485)
0929 19:09:30.166258 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:30.166272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:30.166325 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:30.166457 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:30.166459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:30.166461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:30.166533 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.166535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.167504 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.167505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.168818 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.168821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.169568 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.169569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.169647 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.169651 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:30.169710 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:09:30.169759 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":63}
I20260929 19:09:30.169963 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.166198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:35927 (local address 127.26.78.194:40637)
0929 19:09:30.166417 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:30.166428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:30.166487 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:30.166609 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:30.166609 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:30.166610 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:30.166669 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.166670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.167512 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.167513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.168818 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:30.168821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.169568 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.169569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:30.169647 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.169651 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:30.169709 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:09:30.169759 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":127,"thread_start_us":85,"threads_started":1}
I20260929 19:09:30.170303 12211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa4c0dcd9554497a8066f6604ca4276" candidate_uuid: "d55914dc36104b5b907f72368538e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caedd4d055c5473086ac03fc296c9b2f" is_pre_election: true
I20260929 19:09:30.170370 12473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa4c0dcd9554497a8066f6604ca4276" candidate_uuid: "d55914dc36104b5b907f72368538e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b25c4bfb8725432e875e1134ba8841c8" is_pre_election: true
I20260929 19:09:30.170449 12211 raft_consensus.cc:2454] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55914dc36104b5b907f72368538e755 in term 0.
I20260929 19:09:30.170526 12473 raft_consensus.cc:2454] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55914dc36104b5b907f72368538e755 in term 0.
I20260929 19:09:30.170656 12276 leader_election.cc:302] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caedd4d055c5473086ac03fc296c9b2f, d55914dc36104b5b907f72368538e755; no voters:
I20260929 19:09:30.170795 12540 raft_consensus.cc:2781] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:09:30.170876 12540 raft_consensus.cc:482] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:09:30.170894 12540 raft_consensus.cc:3037] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:30.171615 12540 raft_consensus.cc:504] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.171761 12540 leader_election.cc:288] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [CANDIDATE]: Term 1 election: Requested vote from peers b25c4bfb8725432e875e1134ba8841c8 (127.26.78.195:35927), caedd4d055c5473086ac03fc296c9b2f (127.26.78.193:43329)
I20260929 19:09:30.171916 12211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa4c0dcd9554497a8066f6604ca4276" candidate_uuid: "d55914dc36104b5b907f72368538e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caedd4d055c5473086ac03fc296c9b2f"
I20260929 19:09:30.171979 12211 raft_consensus.cc:3037] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:30.172693 12211 raft_consensus.cc:2454] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55914dc36104b5b907f72368538e755 in term 1.
I20260929 19:09:30.172811 12473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa4c0dcd9554497a8066f6604ca4276" candidate_uuid: "d55914dc36104b5b907f72368538e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b25c4bfb8725432e875e1134ba8841c8"
I20260929 19:09:30.172849 12276 leader_election.cc:302] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caedd4d055c5473086ac03fc296c9b2f, d55914dc36104b5b907f72368538e755; no voters:
I20260929 19:09:30.172895 12473 raft_consensus.cc:3037] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:30.172981 12540 raft_consensus.cc:2781] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:30.173111 12540 raft_consensus.cc:686] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 1 LEADER]: Becoming Leader. State: Replica: d55914dc36104b5b907f72368538e755, State: Running, Role: LEADER
I20260929 19:09:30.173213 12540 consensus_queue.cc:237] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } }
I20260929 19:09:30.173806 12473 raft_consensus.cc:2454] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55914dc36104b5b907f72368538e755 in term 1.
I20260929 19:09:30.174059 12068 catalog_manager.cc:5495] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 reported cstate change: term changed from 0 to 1, leader changed from <none> to d55914dc36104b5b907f72368538e755 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "d55914dc36104b5b907f72368538e755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d55914dc36104b5b907f72368538e755" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 36881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260929 19:09:30.185986 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.183482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.26.78.194:36881)
0929 19:09:30.183541 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:30.183543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:30.183565 (+ 22us) server_negotiation.cc:407] Connection header received
0929 19:09:30.183603 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:30.183604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:30.183626 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:30.183684 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:09:30.183830 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.184323 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.185134 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:30.185340 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:30.185409 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:30.185592 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:09:30.185699 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:09:30.185761 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:30.185802 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 19:09:30.185854 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:09:30.187916 12473 raft_consensus.cc:1261] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 1 FOLLOWER]: Refusing update from remote peer d55914dc36104b5b907f72368538e755: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:30.187934 12211 raft_consensus.cc:1261] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 1 FOLLOWER]: Refusing update from remote peer d55914dc36104b5b907f72368538e755: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:30.188273 12540 consensus_queue.cc:1035] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b25c4bfb8725432e875e1134ba8841c8" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 35927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:30.188442 12543 consensus_queue.cc:1035] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caedd4d055c5473086ac03fc296c9b2f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:30.191680 12547 mvcc.cc:205] Tried to move back new op lower bound from 7334743941886820352 to 7334743941830209536. Current Snapshot: MvccSnapshot[applied={T|T < 7334743941886820352}]
I20260929 19:09:30.193022 12068 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:30.193122 12068 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:30.193917 12068 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb86453008904a11abbfebe4bd6fcf1f: Sending DeleteTablet for 3 replicas of tablet baa4c0dcd9554497a8066f6604ca4276
I20260929 19:09:30.194268 12191 tablet_service.cc:1514] Processing DeleteTablet for tablet baa4c0dcd9554497a8066f6604ca4276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:30 UTC) from {username='slave'} at 127.0.0.1:43736
I20260929 19:09:30.194296 12322 tablet_service.cc:1514] Processing DeleteTablet for tablet baa4c0dcd9554497a8066f6604ca4276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:30 UTC) from {username='slave'} at 127.0.0.1:50852
I20260929 19:09:30.194509 12453 tablet_service.cc:1514] Processing DeleteTablet for tablet baa4c0dcd9554497a8066f6604ca4276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:30 UTC) from {username='slave'} at 127.0.0.1:39616
I20260929 19:09:30.194620 12556 tablet_replica.cc:330] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f: stopping tablet replica
I20260929 19:09:30.194721 12558 tablet_replica.cc:330] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: stopping tablet replica
I20260929 19:09:30.194728 12556 raft_consensus.cc:2229] T baa4c0dcd9554497a8066f6604ca4276 P caedd4d055c5473086ac03fc296c9b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:30.194794 12558 raft_consensus.cc:2229] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:30.194841 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12129
I20260929 19:09:30.194878 12558 raft_consensus.cc:2258] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:30.195712 12557 tablet_replica.cc:330] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: stopping tablet replica
I20260929 19:09:30.195890 12557 raft_consensus.cc:2229] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:09:30.195995 12557 raft_consensus.cc:2258] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:30.196480 12558 ts_tablet_manager.cc:1905] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:30.196681 12557 ts_tablet_manager.cc:1905] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:30.198494 12558 ts_tablet_manager.cc:1918] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:30.198506 12557 ts_tablet_manager.cc:1918] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:30.198573 12558 log.cc:1196] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/baa4c0dcd9554497a8066f6604ca4276
I20260929 19:09:30.198688 12557 log.cc:1196] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/baa4c0dcd9554497a8066f6604ca4276
I20260929 19:09:30.198817 12558 ts_tablet_manager.cc:1939] T baa4c0dcd9554497a8066f6604ca4276 P b25c4bfb8725432e875e1134ba8841c8: Deleting consensus metadata
I20260929 19:09:30.199075 12557 ts_tablet_manager.cc:1939] T baa4c0dcd9554497a8066f6604ca4276 P d55914dc36104b5b907f72368538e755: Deleting consensus metadata
I20260929 19:09:30.199358 12055 catalog_manager.cc:4841] TS b25c4bfb8725432e875e1134ba8841c8 (127.26.78.195:35927): tablet baa4c0dcd9554497a8066f6604ca4276 (table test-table [id=d31131263c7d4cfa947745aca2bd0540]) successfully deleted
I20260929 19:09:30.199496 12056 catalog_manager.cc:4841] TS d55914dc36104b5b907f72368538e755 (127.26.78.194:36881): tablet baa4c0dcd9554497a8066f6604ca4276 (table test-table [id=d31131263c7d4cfa947745aca2bd0540]) successfully deleted
W20260929 19:09:30.201820 12054 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:43329 (error 108)
I20260929 19:09:30.202387 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:30.202238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43329 (local address 127.0.0.1:43748)
0929 19:09:30.202303 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:30.202354 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 19:09:30.202411 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12260
W20260929 19:09:30.203891 12054 catalog_manager.cc:4584] TS caedd4d055c5473086ac03fc296c9b2f (127.26.78.193:43329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet baa4c0dcd9554497a8066f6604ca4276: Network error: Client connection negotiation failed: client connection to 127.26.78.193:43329: connect: Connection refused (error 111)
I20260929 19:09:30.207856 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12391
I20260929 19:09:30.212904 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12038
2026-09-29T19:09:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (691 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:30 dist-test-slave-xz51 krb5kdc[12562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:30 dist-test-slave-xz51 krb5kdc[12562](info): set up 2 sockets
Sep 29 19:09:30 dist-test-slave-xz51 krb5kdc[12562](info): commencing operation
krb5kdc: starting...
W20260929 19:09:31.249872 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:31.369127 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44115
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:31.449486 12575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:31.449738 12575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:31.449774 12575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:31.450975 12575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:31.451051 12575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:31.451078 12575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:31.451112 12575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:31.451134 12575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:31.452625 12575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:31.452904 12575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:31.453153 12575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:31.458119 12583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:31.458209 12581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:31.458240 12580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:31.458580 12575 server_base.cc:1034] running on GCE node
I20260929 19:09:31.458824 12575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:31.459074 12575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:31.460217 12575 hybrid_clock.cc:648] HybridClock initialized: now 1790708971460204 us; error 30 us; skew 500 ppm
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:31.462749 12575 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:31.463135 12575 webserver.cc:460] Webserver started at http://127.26.78.254:34557/ using document root <none> and password file <none>
I20260929 19:09:31.463397 12575 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:31.463466 12575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:31.463637 12575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:31.464726 12575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "532434feaeae4b5f8d090a6229f34a47"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "57c66bc94774e79035a1579688ef0568"
server_key_iv: "321898abb9ef48573fccbf82fba155dc"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.465094 12575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "532434feaeae4b5f8d090a6229f34a47"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "57c66bc94774e79035a1579688ef0568"
server_key_iv: "321898abb9ef48573fccbf82fba155dc"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.466512 12575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:09:31.467249 12590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.467430 12575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:31.467518 12575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "532434feaeae4b5f8d090a6229f34a47"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "57c66bc94774e79035a1579688ef0568"
server_key_iv: "321898abb9ef48573fccbf82fba155dc"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.467599 12575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:31.503374 12575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:31.505731 12575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:31.505939 12575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:31.509598 12575 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:44115
I20260929 19:09:31.509637 12642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:44115 every 8 connection(s)
I20260929 19:09:31.510030 12575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:31.510980 12643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:31.513337 12643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47: Bootstrap starting.
I20260929 19:09:31.514005 12643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:31.514356 12643 log.cc:824] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:31.514444 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12575
I20260929 19:09:31.514559 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:31.514765 26939 external_mini_cluster.cc:1428] Setting key 7dec41e36d5ecdba1f8b7dbca2c52f42
I20260929 19:09:31.515213 12643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47: No bootstrap required, opened a new log
I20260929 19:09:31.516925 12643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "532434feaeae4b5f8d090a6229f34a47" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44115 } }
I20260929 19:09:31.517094 12643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:31.517133 12643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 532434feaeae4b5f8d090a6229f34a47, State: Initialized, Role: FOLLOWER
I20260929 19:09:31.517306 12643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "532434feaeae4b5f8d090a6229f34a47" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44115 } }
I20260929 19:09:31.517402 12643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:31.517439 12643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:31.517489 12643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:31.518527 12643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "532434feaeae4b5f8d090a6229f34a47" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44115 } }
I20260929 19:09:31.518617 12643 leader_election.cc:302] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 532434feaeae4b5f8d090a6229f34a47; no voters:
I20260929 19:09:31.518811 12643 leader_election.cc:288] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:31.518858 12648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:31.519017 12648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [term 1 LEADER]: Becoming Leader. State: Replica: 532434feaeae4b5f8d090a6229f34a47, State: Running, Role: LEADER
I20260929 19:09:31.519099 12643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:31.519183 12648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "532434feaeae4b5f8d090a6229f34a47" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44115 } }
I20260929 19:09:31.519626 12649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 532434feaeae4b5f8d090a6229f34a47. Latest consensus state: current_term: 1 leader_uuid: "532434feaeae4b5f8d090a6229f34a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "532434feaeae4b5f8d090a6229f34a47" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44115 } } }
I20260929 19:09:31.519781 12649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:31.520087 12657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:31.520592 12650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "532434feaeae4b5f8d090a6229f34a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "532434feaeae4b5f8d090a6229f34a47" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44115 } } }
I20260929 19:09:31.520684 12650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:31.520627 12657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:31.522300 12657 catalog_manager.cc:1347] Generated new cluster ID: aa39bf540f1146bcbcbe43dc1783ca5a
I20260929 19:09:31.522356 12657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:31.527812 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:31.515275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.26.78.254:44115)
0929 19:09:31.515478 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:31.515486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:31.515511 (+ 25us) server_negotiation.cc:407] Connection header received
0929 19:09:31.516904 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:31.516911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:31.516960 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:31.517077 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:31.517944 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.519079 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.520302 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.520981 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.521091 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:31.523709 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:31.523713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:31.523713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:31.523726 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:31.526174 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:31.526561 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:31.526565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:31.526565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:31.526692 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:31.526907 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:31.526908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:31.526908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:31.527162 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:31.527344 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:31.527447 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 19:09:31.527593 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260929 19:09:31.528703 12657 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:31.529174 12657 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:31.532922 12657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47: Generated new TSK 0
I20260929 19:09:31.533100 12657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:09:31.568548 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44115
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:31.670739 12669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:31.671000 12669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:31.671028 12669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:31.672259 12669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:31.672300 12669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:31.672328 12669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:09:31.673813 12669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:31.674129 12669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:31.674398 12669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:31.680224 12674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:31.680265 12675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:31.680574 12669 server_base.cc:1034] running on GCE node
W20260929 19:09:31.680378 12677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:31.680853 12669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:31.681149 12669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:31.682305 12669 hybrid_clock.cc:648] HybridClock initialized: now 1790708971682292 us; error 36 us; skew 500 ppm
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:31.684947 12669 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:09:31.685389 12669 webserver.cc:460] Webserver started at http://127.26.78.193:34171/ using document root <none> and password file <none>
I20260929 19:09:31.685626 12669 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:31.685698 12669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:31.685906 12669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:31.686985 12669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "a27a540a0e064400ae0e1c85e02fe49f"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "449b6b6236a9d05fbe9c388ae604d65f"
server_key_iv: "87be58859828cdcf756b993a322f44e1"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.687362 12669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "a27a540a0e064400ae0e1c85e02fe49f"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "449b6b6236a9d05fbe9c388ae604d65f"
server_key_iv: "87be58859828cdcf756b993a322f44e1"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.688745 12669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:31.689512 12684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.689720 12669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:31.689824 12669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "a27a540a0e064400ae0e1c85e02fe49f"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "449b6b6236a9d05fbe9c388ae604d65f"
server_key_iv: "87be58859828cdcf756b993a322f44e1"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.689954 12669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:31.703063 12669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:31.705053 12669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:31.705226 12669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:31.705478 12669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:31.705832 12669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:31.705897 12669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.705940 12669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:31.705986 12669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.711951 12669 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:43611
I20260929 19:09:31.712002 12797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:43611 every 8 connection(s)
I20260929 19:09:31.712311 12669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:09:31.714392 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12669
I20260929 19:09:31.714488 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:09:31.714699 26939 external_mini_cluster.cc:1428] Setting key 6eb141481c83fa7594b612a0cc2efc75
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
I20260929 19:09:31.722496 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:31.713006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:33219 (local address 127.26.78.254:44115)
0929 19:09:31.713050 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:31.713053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:31.713259 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:09:31.713792 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:31.713794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:31.713811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:31.713906 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:31.715117 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.715724 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.716737 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.716966 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.717054 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:31.719475 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:31.719476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:31.719476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:31.719484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:31.721036 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:31.721456 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:31.721458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:31.721458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:31.721516 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:31.721778 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:31.721779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:31.721780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:31.721934 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:31.722020 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:31.722301 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 19:09:31.722407 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:09:31.722676 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:31.712965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44115 (local address 127.26.78.193:33219)
0929 19:09:31.713140 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:31.713175 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:31.713682 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:31.713995 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:31.713999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:31.714682 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:31.715012 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:31.715019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.715820 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.715821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:31.716641 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:31.716643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.717064 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.717065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:31.717203 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:31.717740 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:31.717749 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:31.719290 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:31.721129 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:31.721137 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:31.721139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:31.721376 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:31.721594 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:31.721595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:31.721595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:31.721709 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:31.722025 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:31.722032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:31.722170 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 19:09:31.722280 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 19:09:31.722438 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260929 19:09:31.723227 12798 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44115
I20260929 19:09:31.723323 12798 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:31.723518 12798 heartbeater.cc:511] Master 127.26.78.254:44115 requested a full tablet report, sending...
I20260929 19:09:31.724049 12607 ts_manager.cc:194] Registered new tserver with Master: a27a540a0e064400ae0e1c85e02fe49f (127.26.78.193:43611)
I20260929 19:09:31.724649 12607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:33219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:09:31.740235 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44115
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:31.823024 12803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:31.823239 12803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:31.823263 12803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:31.824393 12803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:31.824427 12803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:31.824452 12803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:09:31.825985 12803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:31.826184 12803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:31.826370 12803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:31.831486 12808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:31.831496 12811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:31.831496 12809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:31.831594 12803 server_base.cc:1034] running on GCE node
I20260929 19:09:31.831951 12803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:31.832181 12803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:31.833320 12803 hybrid_clock.cc:648] HybridClock initialized: now 1790708971833305 us; error 27 us; skew 500 ppm
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:31.835803 12803 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:09:31.836288 12803 webserver.cc:460] Webserver started at http://127.26.78.194:34717/ using document root <none> and password file <none>
I20260929 19:09:31.836537 12803 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:31.836606 12803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:31.836758 12803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:31.837795 12803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "359cb592c8544fee9bcd1a725fab1228"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "d959dbdd73e2e9f2224d35e03d938a62"
server_key_iv: "7e624c64c8bb3bac381eae9d0c954e0b"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.838227 12803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "359cb592c8544fee9bcd1a725fab1228"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "d959dbdd73e2e9f2224d35e03d938a62"
server_key_iv: "7e624c64c8bb3bac381eae9d0c954e0b"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.839532 12803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:31.840237 12818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.840415 12803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:31.840515 12803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "359cb592c8544fee9bcd1a725fab1228"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "d959dbdd73e2e9f2224d35e03d938a62"
server_key_iv: "7e624c64c8bb3bac381eae9d0c954e0b"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.840601 12803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:31.853557 12803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:31.855522 12803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:31.855688 12803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:31.855927 12803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:31.856238 12803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:31.856293 12803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.856328 12803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:31.856355 12803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.862267 12803 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:46093
I20260929 19:09:31.862368 12931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:46093 every 8 connection(s)
I20260929 19:09:31.862675 12803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:09:31.865337 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12803
I20260929 19:09:31.865449 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:09:31.865650 26939 external_mini_cluster.cc:1428] Setting key f373f1f759c8c3d808671fca17b9a048
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:31.873059 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:31.863375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:40731 (local address 127.26.78.254:44115)
0929 19:09:31.863418 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:31.863420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:31.863639 (+ 219us) server_negotiation.cc:407] Connection header received
0929 19:09:31.864186 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:31.864187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:31.864198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:31.864266 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:31.865173 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.866006 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.867150 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.867479 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.867554 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:31.870043 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:31.870044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:31.870045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:31.870053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:31.871590 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:31.872030 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:31.872031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:31.872032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:31.872094 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:31.872394 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:31.872395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:31.872396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:31.872513 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:31.872590 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:31.872871 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 19:09:31.872963 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:09:31.873263 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:31.863359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44115 (local address 127.26.78.194:40731)
0929 19:09:31.863508 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:31.863543 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:31.864086 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:31.864317 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:31.864320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:31.864879 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:31.865074 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:31.865078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.866103 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.866105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:31.867052 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:31.867055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:31.867562 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:31.867563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:31.867733 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:31.868285 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:31.868298 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:31.869941 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:31.871685 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:31.871693 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:31.871695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:31.871935 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:31.872189 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:31.872190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:31.872191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:31.872312 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:31.872581 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:31.872588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:31.872736 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 19:09:31.872858 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 19:09:31.873023 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260929 19:09:31.873780 12932 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44115
I20260929 19:09:31.873900 12932 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:31.874085 12932 heartbeater.cc:511] Master 127.26.78.254:44115 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
I20260929 19:09:31.874488 12607 ts_manager.cc:194] Registered new tserver with Master: 359cb592c8544fee9bcd1a725fab1228 (127.26.78.194:46093)
I20260929 19:09:31.874905 12607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:40731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:09:31.892726 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44115
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:31.975241 12937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:31.975448 12937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:31.975472 12937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:31.976595 12937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:31.976634 12937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:31.976660 12937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:09:31.978207 12937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:44115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:31.978410 12937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:31.978578 12937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:31.983547 12943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:31.983548 12945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:31.983649 12937 server_base.cc:1034] running on GCE node
W20260929 19:09:31.983572 12942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:31.984112 12937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:31.984401 12937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:31.985563 12937 hybrid_clock.cc:648] HybridClock initialized: now 1790708971985538 us; error 38 us; skew 500 ppm
Sep 29 19:09:31 dist-test-slave-xz51 krb5kdc[12562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:31.988045 12937 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:09:31.988497 12937 webserver.cc:460] Webserver started at http://127.26.78.195:45341/ using document root <none> and password file <none>
I20260929 19:09:31.988741 12937 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:31.988812 12937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:31.988973 12937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:31.990113 12937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "88c8cb2b40fa96a964ccbff49d7c2291"
server_key_iv: "b39bf4d174f4e422bfd6493b223a5f10"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.990480 12937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "88c8cb2b40fa96a964ccbff49d7c2291"
server_key_iv: "b39bf4d174f4e422bfd6493b223a5f10"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.991802 12937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:31.992516 12952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:31.992686 12937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:31.992828 12937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57"
format_stamp: "Formatted at 2026-09-29 19:09:31 on dist-test-slave-xz51"
server_key: "88c8cb2b40fa96a964ccbff49d7c2291"
server_key_iv: "b39bf4d174f4e422bfd6493b223a5f10"
server_key_version: "encryptionkey@0"
I20260929 19:09:31.992926 12937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:32.003770 12937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:32.005606 12937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:32.005786 12937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:32.006098 12937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:32.006428 12937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:32.006484 12937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.006524 12937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:32.006552 12937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.012594 12937 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:39965
I20260929 19:09:32.012643 13065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:39965 every 8 connection(s)
I20260929 19:09:32.012966 12937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:09:32.017715 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 12937
I20260929 19:09:32.017851 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:09:32.018070 26939 external_mini_cluster.cc:1428] Setting key a2e2e1016ad0bc834ee695deb75608bb
Sep 29 19:09:32 dist-test-slave-xz51 krb5kdc[12562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708971, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:32.022276 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.013630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:46085 (local address 127.26.78.254:44115)
0929 19:09:32.013675 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.013677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.013900 (+ 223us) server_negotiation.cc:407] Connection header received
0929 19:09:32.014382 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.014383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.014394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.014444 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:32.015348 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.015846 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.016801 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.016985 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.017035 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.019541 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:32.019542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:32.019542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:32.019549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:32.020972 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:32.021429 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:32.021430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:32.021431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:32.021491 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:32.021701 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:32.021702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:32.021702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:32.021804 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:32.021921 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.022146 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.022210 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:09:32.022509 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.013625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44115 (local address 127.26.78.195:46085)
0929 19:09:32.013766 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.013801 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.014292 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.014506 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.014510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.015076 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:32.015269 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.015274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.015937 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.015937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.016728 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.016729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.017057 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.017058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.017164 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.017652 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:32.017664 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:32.019455 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:32.021059 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:32.021066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:32.021069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:32.021350 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:32.021546 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:32.021547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:32.021547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:32.021649 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:32.021915 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:32.021919 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:32.022036 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.022119 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.022240 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260929 19:09:32.023169 13066 heartbeater.cc:347] Connected to a master server at 127.26.78.254:44115
I20260929 19:09:32.023278 13066 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:32.023466 13066 heartbeater.cc:511] Master 127.26.78.254:44115 requested a full tablet report, sending...
I20260929 19:09:32.023813 12607 ts_manager.cc:194] Registered new tserver with Master: 0cf1d6f3c4be4dd38447c6e908a9fd57 (127.26.78.195:39965)
I20260929 19:09:32.024199 12607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:46085
I20260929 19:09:32.030160 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:09:32.038360 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.031533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.26.78.254:44115)
0929 19:09:32.031630 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.031632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.031727 (+ 95us) server_negotiation.cc:407] Connection header received
0929 19:09:32.031818 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.031819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.031830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.031911 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:32.032554 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.033367 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.034227 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.034437 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.034529 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.035844 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:32.035846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:32.035846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:32.035854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:32.037029 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:32.037438 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:32.037440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:32.037441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:32.037496 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:32.037735 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:32.037736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:32.037736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:32.037956 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:32.038071 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.038196 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.038259 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 19:09:32.040043 12607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:09:32.041538 12607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:09:32.048018 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.043856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43611 (local address 127.0.0.1:51482)
0929 19:09:32.044180 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.044223 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.044368 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.044597 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.044600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.044603 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.044675 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.044677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.045673 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.045675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.046799 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.046802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.047695 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.047696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.047781 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.047795 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.047873 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.047924 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":91,"threads_started":1}
I20260929 19:09:32.048031 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.044065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51482 (local address 127.26.78.193:43611)
0929 19:09:32.044223 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.044228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.044324 (+ 96us) server_negotiation.cc:407] Connection header received
0929 19:09:32.044460 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.044463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.044478 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.044574 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.044765 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.045574 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.046893 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.047614 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.047681 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.047744 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.047904 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.047948 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260929 19:09:32.048394 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.044073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:39965 (local address 127.0.0.1:60002)
0929 19:09:32.044273 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.044284 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.044343 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.044943 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.044944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.044946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.044983 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.044985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.046271 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.046272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.047259 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.047261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.048160 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.048161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.048220 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.048225 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.048281 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.048318 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":97,"thread_start_us":76,"threads_started":1}
I20260929 19:09:32.048446 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.044249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.26.78.195:39965)
0929 19:09:32.044425 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.044430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.044439 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:09:32.044495 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.044497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.044513 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.044585 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.045049 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.046164 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.047345 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.047966 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.048027 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.048078 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.048317 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.048365 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260929 19:09:32.048807 12732 tablet_service.cc:1467] Processing CreateTablet for tablet fcaaebff7e5f41bca03e85dcb704c77b (DEFAULT_TABLE table=test-table [id=adb0ca3c175a415d8c758a02b9fb6e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:32.049194 12732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcaaebff7e5f41bca03e85dcb704c77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:32.049127 13000 tablet_service.cc:1467] Processing CreateTablet for tablet fcaaebff7e5f41bca03e85dcb704c77b (DEFAULT_TABLE table=test-table [id=adb0ca3c175a415d8c758a02b9fb6e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:32.049465 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.043873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:46093 (local address 127.0.0.1:45162)
0929 19:09:32.044202 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.044234 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.044344 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.044869 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.044869 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.044871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.044904 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.044905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.046282 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.046283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.047525 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.047528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.049127 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.049128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.049192 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.049196 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.049360 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.049402 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":22,"thread_start_us":47,"threads_started":1}
I20260929 19:09:32.049474 13000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcaaebff7e5f41bca03e85dcb704c77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:32.049736 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.043960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.26.78.194:46093)
0929 19:09:32.044712 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.044717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.044727 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:09:32.044763 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.044765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.044779 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.044866 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.044986 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.046155 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.047631 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.049021 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.049096 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.049149 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.049606 (+ 457us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.049651 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":118,"threads_started":1}
I20260929 19:09:32.050614 12866 tablet_service.cc:1467] Processing CreateTablet for tablet fcaaebff7e5f41bca03e85dcb704c77b (DEFAULT_TABLE table=test-table [id=adb0ca3c175a415d8c758a02b9fb6e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:32.050946 12866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcaaebff7e5f41bca03e85dcb704c77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:32.052114 13083 tablet_bootstrap.cc:492] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Bootstrap starting.
I20260929 19:09:32.052784 13083 tablet_bootstrap.cc:654] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:32.053174 13083 log.cc:824] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:32.053928 13083 tablet_bootstrap.cc:492] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: No bootstrap required, opened a new log
I20260929 19:09:32.053993 13083 ts_tablet_manager.cc:1397] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:09:32.054036 13086 tablet_bootstrap.cc:492] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Bootstrap starting.
I20260929 19:09:32.054224 13084 tablet_bootstrap.cc:492] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f: Bootstrap starting.
I20260929 19:09:32.054669 13086 tablet_bootstrap.cc:654] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:32.054807 13084 tablet_bootstrap.cc:654] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:32.055438 13086 log.cc:824] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:32.055506 13084 log.cc:824] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:32.056206 13086 tablet_bootstrap.cc:492] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: No bootstrap required, opened a new log
I20260929 19:09:32.056280 13086 ts_tablet_manager.cc:1397] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:09:32.056303 13084 tablet_bootstrap.cc:492] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f: No bootstrap required, opened a new log
I20260929 19:09:32.056353 13084 ts_tablet_manager.cc:1397] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:09:32.056442 13083 raft_consensus.cc:348] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.056617 13083 raft_consensus.cc:374] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:32.056660 13083 raft_consensus.cc:729] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf1d6f3c4be4dd38447c6e908a9fd57, State: Initialized, Role: FOLLOWER
I20260929 19:09:32.056787 13083 consensus_queue.cc:260] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.057070 13083 ts_tablet_manager.cc:1428] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:32.057204 13066 heartbeater.cc:503] Master 127.26.78.254:44115 was elected leader, sending a full tablet report...
I20260929 19:09:32.058267 13084 raft_consensus.cc:348] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.058249 13086 raft_consensus.cc:348] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.058439 13086 raft_consensus.cc:374] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:32.058434 13084 raft_consensus.cc:374] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:32.058472 13086 raft_consensus.cc:729] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359cb592c8544fee9bcd1a725fab1228, State: Initialized, Role: FOLLOWER
I20260929 19:09:32.058472 13084 raft_consensus.cc:729] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a27a540a0e064400ae0e1c85e02fe49f, State: Initialized, Role: FOLLOWER
I20260929 19:09:32.058594 13086 consensus_queue.cc:260] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.058665 13084 consensus_queue.cc:260] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.058899 13086 ts_tablet_manager.cc:1428] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:32.058928 13084 ts_tablet_manager.cc:1428] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:32.058914 12932 heartbeater.cc:503] Master 127.26.78.254:44115 was elected leader, sending a full tablet report...
I20260929 19:09:32.059044 12798 heartbeater.cc:503] Master 127.26.78.254:44115 was elected leader, sending a full tablet report...
W20260929 19:09:32.113461 12933 tablet.cc:2369] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:09:32.213038 12799 tablet.cc:2369] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:09:32.259665 13090 raft_consensus.cc:482] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:32.259795 13090 raft_consensus.cc:504] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.260128 13090 leader_election.cc:288] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a27a540a0e064400ae0e1c85e02fe49f (127.26.78.193:43611), 0cf1d6f3c4be4dd38447c6e908a9fd57 (127.26.78.195:39965)
I20260929 19:09:32.262996 13089 raft_consensus.cc:482] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:32.263144 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.260374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43611 (local address 127.26.78.194:37035)
0929 19:09:32.260443 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.260458 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.260531 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.260708 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.260710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.260711 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.260782 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.260786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.261459 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.261461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.262302 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.262304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.262853 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.262855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.262963 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.262967 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.263026 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.263074 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:09:32.263167 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.260463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:37035 (local address 127.26.78.193:43611)
0929 19:09:32.260510 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.260512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.260523 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:09:32.260624 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.260626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.260637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.260691 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.260872 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.261343 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.262401 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.262753 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.262842 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.262851 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.263045 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.263095 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:09:32.263126 13089 raft_consensus.cc:504] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.263399 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.260617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:44815 (local address 127.26.78.195:39965)
0929 19:09:32.260678 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.260681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.260765 (+ 84us) server_negotiation.cc:407] Connection header received
0929 19:09:32.260831 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.260833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.260842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.260934 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.261090 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.261558 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.262565 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.262990 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.263076 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.263082 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.263286 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.263327 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:09:32.263456 12752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaaebff7e5f41bca03e85dcb704c77b" candidate_uuid: "359cb592c8544fee9bcd1a725fab1228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a27a540a0e064400ae0e1c85e02fe49f" is_pre_election: true
I20260929 19:09:32.263542 13089 leader_election.cc:288] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 359cb592c8544fee9bcd1a725fab1228 (127.26.78.194:46093), a27a540a0e064400ae0e1c85e02fe49f (127.26.78.193:43611)
I20260929 19:09:32.263602 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.260505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:39965 (local address 127.26.78.194:44815)
0929 19:09:32.260695 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.260704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.260778 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.260940 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.260941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.260943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.261018 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.261019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.261653 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.261654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.262474 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.262476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.263116 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.263117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.263196 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.263199 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.263496 (+ 297us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.263538 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":120,"threads_started":1}
I20260929 19:09:32.263603 12752 raft_consensus.cc:2454] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 359cb592c8544fee9bcd1a725fab1228 in term 0.
W20260929 19:09:32.263546 13067 tablet.cc:2369] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:09:32.264011 12822 leader_election.cc:302] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 359cb592c8544fee9bcd1a725fab1228, a27a540a0e064400ae0e1c85e02fe49f; no voters:
I20260929 19:09:32.264137 13090 raft_consensus.cc:2781] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:09:32.264186 13090 raft_consensus.cc:482] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:09:32.264232 13090 raft_consensus.cc:3037] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:32.264358 13020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaaebff7e5f41bca03e85dcb704c77b" candidate_uuid: "359cb592c8544fee9bcd1a725fab1228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" is_pre_election: true
I20260929 19:09:32.264447 13020 raft_consensus.cc:2454] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 359cb592c8544fee9bcd1a725fab1228 in term 0.
I20260929 19:09:32.265148 13090 raft_consensus.cc:504] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.265270 13090 leader_election.cc:288] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [CANDIDATE]: Term 1 election: Requested vote from peers a27a540a0e064400ae0e1c85e02fe49f (127.26.78.193:43611), 0cf1d6f3c4be4dd38447c6e908a9fd57 (127.26.78.195:39965)
I20260929 19:09:32.265502 12752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaaebff7e5f41bca03e85dcb704c77b" candidate_uuid: "359cb592c8544fee9bcd1a725fab1228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a27a540a0e064400ae0e1c85e02fe49f"
I20260929 19:09:32.265511 13020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaaebff7e5f41bca03e85dcb704c77b" candidate_uuid: "359cb592c8544fee9bcd1a725fab1228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57"
I20260929 19:09:32.265576 13020 raft_consensus.cc:3037] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:32.265579 12752 raft_consensus.cc:3037] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:32.266512 13020 raft_consensus.cc:2454] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 359cb592c8544fee9bcd1a725fab1228 in term 1.
I20260929 19:09:32.266515 12752 raft_consensus.cc:2454] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 359cb592c8544fee9bcd1a725fab1228 in term 1.
I20260929 19:09:32.266721 12822 leader_election.cc:302] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf1d6f3c4be4dd38447c6e908a9fd57, 359cb592c8544fee9bcd1a725fab1228; no voters:
I20260929 19:09:32.266829 13090 raft_consensus.cc:2781] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:32.266947 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.263725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:46093 (local address 127.26.78.195:39483)
0929 19:09:32.263780 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.263791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.263828 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.264003 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.264004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.264006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.264073 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.264075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.264927 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.264928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.265918 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.265920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.266670 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.266671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.266757 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.266760 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.266832 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.266881 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 19:09:32.267009 13090 raft_consensus.cc:686] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 1 LEADER]: Becoming Leader. State: Replica: 359cb592c8544fee9bcd1a725fab1228, State: Running, Role: LEADER
I20260929 19:09:32.267125 13090 consensus_queue.cc:237] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } }
I20260929 19:09:32.267192 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.263833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:39483 (local address 127.26.78.194:46093)
0929 19:09:32.263870 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.263872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.263880 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:09:32.263901 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.263902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.263911 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.263962 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.264138 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.264857 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.266015 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.266561 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.266634 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.266642 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.267097 (+ 455us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.267135 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:09:32.267433 12886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaaebff7e5f41bca03e85dcb704c77b" candidate_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359cb592c8544fee9bcd1a725fab1228" is_pre_election: true
I20260929 19:09:32.267740 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.263796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43611 (local address 127.26.78.195:45981)
0929 19:09:32.264048 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.264057 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.264607 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.264776 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.264776 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.264778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.264833 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.264834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.265646 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.265647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.266788 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.266792 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.267493 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.267511 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.267578 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.267582 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.267633 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.267669 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":88,"threads_started":1}
I20260929 19:09:32.267939 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.263956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:45981 (local address 127.26.78.193:43611)
0929 19:09:32.263989 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.263991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.264103 (+ 112us) server_negotiation.cc:407] Connection header received
0929 19:09:32.264696 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.264697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.264705 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.264730 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.264919 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.265560 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.266895 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.267415 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.267480 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.267488 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.267839 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.267888 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:09:32.268173 12752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaaebff7e5f41bca03e85dcb704c77b" candidate_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a27a540a0e064400ae0e1c85e02fe49f" is_pre_election: true
I20260929 19:09:32.268296 12752 raft_consensus.cc:2379] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0cf1d6f3c4be4dd38447c6e908a9fd57 in current term 1: Already voted for candidate 359cb592c8544fee9bcd1a725fab1228 in this term.
I20260929 19:09:32.268537 12956 leader_election.cc:302] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cf1d6f3c4be4dd38447c6e908a9fd57; no voters: 359cb592c8544fee9bcd1a725fab1228, a27a540a0e064400ae0e1c85e02fe49f
I20260929 19:09:32.268493 12607 catalog_manager.cc:5495] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 reported cstate change: term changed from 0 to 1, leader changed from <none> to 359cb592c8544fee9bcd1a725fab1228 (127.26.78.194). New cstate: current_term: 1 leader_uuid: "359cb592c8544fee9bcd1a725fab1228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359cb592c8544fee9bcd1a725fab1228" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 46093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260929 19:09:32.268668 13089 raft_consensus.cc:2726] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 19:09:32.277901 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.275503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.26.78.194:46093)
0929 19:09:32.275543 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.275545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.275571 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:09:32.275613 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.275614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.275629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.275705 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:09:32.275805 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.276262 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.277038 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.277230 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.277365 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.277501 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:09:32.277597 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:09:32.277708 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.277739 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.277749 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:09:32.279640 12752 raft_consensus.cc:1261] T fcaaebff7e5f41bca03e85dcb704c77b P a27a540a0e064400ae0e1c85e02fe49f [term 1 FOLLOWER]: Refusing update from remote peer 359cb592c8544fee9bcd1a725fab1228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:32.279659 13020 raft_consensus.cc:1261] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 1 FOLLOWER]: Refusing update from remote peer 359cb592c8544fee9bcd1a725fab1228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:32.279976 13090 consensus_queue.cc:1035] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf1d6f3c4be4dd38447c6e908a9fd57" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:32.280138 13094 consensus_queue.cc:1035] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a27a540a0e064400ae0e1c85e02fe49f" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:32.282958 13097 mvcc.cc:205] Tried to move back new op lower bound from 7334743950454767616 to 7334743950406561792. Current Snapshot: MvccSnapshot[applied={T|T < 7334743950454767616 or (T in {7334743950454767616})}]
I20260929 19:09:32.283071 13098 mvcc.cc:205] Tried to move back new op lower bound from 7334743950454767616 to 7334743950406561792. Current Snapshot: MvccSnapshot[applied={T|T < 7334743950454767616}]
I20260929 19:09:32.284650 12607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:32.284741 12607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:32.285511 12607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 532434feaeae4b5f8d090a6229f34a47: Sending DeleteTablet for 3 replicas of tablet fcaaebff7e5f41bca03e85dcb704c77b
I20260929 19:09:32.285800 12866 tablet_service.cc:1514] Processing DeleteTablet for tablet fcaaebff7e5f41bca03e85dcb704c77b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:32 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:45162
I20260929 19:09:32.285883 12732 tablet_service.cc:1514] Processing DeleteTablet for tablet fcaaebff7e5f41bca03e85dcb704c77b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:32 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:51482
I20260929 19:09:32.285928 13000 tablet_service.cc:1514] Processing DeleteTablet for tablet fcaaebff7e5f41bca03e85dcb704c77b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:32 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:60002
I20260929 19:09:32.286231 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12669
I20260929 19:09:32.289971 13107 tablet_replica.cc:330] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: stopping tablet replica
I20260929 19:09:32.290084 13107 raft_consensus.cc:2229] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:09:32.290221 13107 raft_consensus.cc:2258] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:32.290396 13109 tablet_replica.cc:330] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: stopping tablet replica
I20260929 19:09:32.290498 13109 raft_consensus.cc:2229] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:32.290601 13109 raft_consensus.cc:2258] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:32.290885 13107 ts_tablet_manager.cc:1905] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:32.291065 13109 ts_tablet_manager.cc:1905] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:09:32.292824 12594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:43611 (error 108)
I20260929 19:09:32.293111 13109 ts_tablet_manager.cc:1918] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:32.293210 13109 log.cc:1196] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/fcaaebff7e5f41bca03e85dcb704c77b
I20260929 19:09:32.293339 13107 ts_tablet_manager.cc:1918] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:32.293475 13107 log.cc:1196] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/fcaaebff7e5f41bca03e85dcb704c77b
I20260929 19:09:32.293470 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12803
I20260929 19:09:32.293792 13109 ts_tablet_manager.cc:1939] T fcaaebff7e5f41bca03e85dcb704c77b P 0cf1d6f3c4be4dd38447c6e908a9fd57: Deleting consensus metadata
I20260929 19:09:32.293936 13107 ts_tablet_manager.cc:1939] T fcaaebff7e5f41bca03e85dcb704c77b P 359cb592c8544fee9bcd1a725fab1228: Deleting consensus metadata
I20260929 19:09:32.294297 12594 catalog_manager.cc:4841] TS 0cf1d6f3c4be4dd38447c6e908a9fd57 (127.26.78.195:39965): tablet fcaaebff7e5f41bca03e85dcb704c77b (table test-table [id=adb0ca3c175a415d8c758a02b9fb6e04]) successfully deleted
I20260929 19:09:32.294601 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.294467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:43611 (local address 127.0.0.1:51484)
0929 19:09:32.294527 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.294565 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:43611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 19:09:32.294723 12594 catalog_manager.cc:4584] TS a27a540a0e064400ae0e1c85e02fe49f (127.26.78.193:43611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcaaebff7e5f41bca03e85dcb704c77b: Network error: Client connection negotiation failed: client connection to 127.26.78.193:43611: connect: Connection refused (error 111)
I20260929 19:09:32.300518 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12937
I20260929 19:09:32.300549 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.300379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:46093 (local address 127.0.0.1:45188)
0929 19:09:32.300471 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.300508 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.194:46093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 19:09:32.305292 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 12575
2026-09-29T19:09:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2092 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T19:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:32Z Disabled control of system clock
I20260929 19:09:32.332386 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43319
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:39127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:43319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:09:32.413571 13114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:32.413842 13114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:32.413879 13114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:32.415037 13114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:32.415074 13114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:32.415087 13114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:32.415100 13114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:32.415112 13114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:32.416499 13114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:39127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:43319
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:32.416706 13114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:32.416947 13114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:09:32.421886 13114 server_base.cc:1034] running on GCE node
W20260929 19:09:32.421878 13119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.421855 13120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.421867 13122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:32.422367 13114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:32.422632 13114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:32.423779 13114 hybrid_clock.cc:648] HybridClock initialized: now 1790708972423765 us; error 31 us; skew 500 ppm
I20260929 19:09:32.425114 13114 webserver.cc:460] Webserver started at http://127.26.78.254:37937/ using document root <none> and password file <none>
I20260929 19:09:32.425390 13114 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:32.425439 13114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:32.425580 13114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:32.426733 13114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "eb054358f2a14c6fa54dbf5b19583342"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "05f3ec4942131cfb18cdd096166b7a98"
server_key_iv: "f2ace44e6cc41e79d1bae43bc3e9a426"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.427111 13114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "eb054358f2a14c6fa54dbf5b19583342"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "05f3ec4942131cfb18cdd096166b7a98"
server_key_iv: "f2ace44e6cc41e79d1bae43bc3e9a426"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.428503 13114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:32.429266 13128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.429445 13114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:32.429553 13114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "eb054358f2a14c6fa54dbf5b19583342"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "05f3ec4942131cfb18cdd096166b7a98"
server_key_iv: "f2ace44e6cc41e79d1bae43bc3e9a426"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.429630 13114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:32.451351 13114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:32.451735 13114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:32.451911 13114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:32.455927 13114 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:43319
I20260929 19:09:32.455981 13180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:43319 every 8 connection(s)
I20260929 19:09:32.456436 13114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:32.456681 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 13114
I20260929 19:09:32.456797 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:32.457005 26939 external_mini_cluster.cc:1428] Setting key 2fd9c663683936d132e7fabc3c4150b2
I20260929 19:09:32.457677 13181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:32.460845 13181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342: Bootstrap starting.
I20260929 19:09:32.461445 13181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:32.461768 13181 log.cc:824] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:32.462621 13181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342: No bootstrap required, opened a new log
I20260929 19:09:32.463052 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.457565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.26.78.254:43319)
0929 19:09:32.457803 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.457813 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.457872 (+ 59us) server_negotiation.cc:407] Connection header received
0929 19:09:32.458443 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.458452 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.458537 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.458692 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:32.458902 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.460216 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.461032 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.461454 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.461594 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.462372 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:32.462376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:32.462376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:32.462533 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:32.462563 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:32.462573 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:32.462624 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.462778 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.462872 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260929 19:09:32.464653 13181 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb054358f2a14c6fa54dbf5b19583342" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43319 } }
I20260929 19:09:32.464787 13181 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:32.464809 13181 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb054358f2a14c6fa54dbf5b19583342, State: Initialized, Role: FOLLOWER
I20260929 19:09:32.464937 13181 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb054358f2a14c6fa54dbf5b19583342" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43319 } }
I20260929 19:09:32.465014 13181 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:32.465037 13181 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:32.465101 13181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:32.465633 13181 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb054358f2a14c6fa54dbf5b19583342" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43319 } }
I20260929 19:09:32.465745 13181 leader_election.cc:302] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb054358f2a14c6fa54dbf5b19583342; no voters:
I20260929 19:09:32.465935 13181 leader_election.cc:288] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:32.466050 13186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:32.466198 13186 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [term 1 LEADER]: Becoming Leader. State: Replica: eb054358f2a14c6fa54dbf5b19583342, State: Running, Role: LEADER
I20260929 19:09:32.466243 13181 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:32.466298 13186 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb054358f2a14c6fa54dbf5b19583342" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43319 } }
I20260929 19:09:32.466636 13186 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb054358f2a14c6fa54dbf5b19583342. Latest consensus state: current_term: 1 leader_uuid: "eb054358f2a14c6fa54dbf5b19583342" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb054358f2a14c6fa54dbf5b19583342" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43319 } } }
I20260929 19:09:32.466712 13186 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:32.466671 13187 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb054358f2a14c6fa54dbf5b19583342" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb054358f2a14c6fa54dbf5b19583342" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43319 } } }
I20260929 19:09:32.466759 13187 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:32.466925 13193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:32.467687 13193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:32.469333 13193 catalog_manager.cc:1347] Generated new cluster ID: a993a988df2944b09d38e8b63c608ec7
I20260929 19:09:32.469388 13193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:32.475000 13193 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:32.475528 13193 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:32.481946 13193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342: Generated new TSK 0
I20260929 19:09:32.482164 13193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:32.488880 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43319
--builtin_ntp_servers=127.26.78.212:39127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:09:32.575790 13205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:32.576037 13205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:32.576073 13205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:32.577237 13205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:32.577306 13205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:32.577346 13205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:09:32.578804 13205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:39127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:32.579070 13205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:32.579378 13205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:32.585340 13210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.585317 13211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.585382 13213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:32.585657 13205 server_base.cc:1034] running on GCE node
I20260929 19:09:32.586004 13205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:32.586299 13205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:32.587473 13205 hybrid_clock.cc:648] HybridClock initialized: now 1790708972587449 us; error 38 us; skew 500 ppm
I20260929 19:09:32.593745 13205 webserver.cc:460] Webserver started at http://127.26.78.193:33397/ using document root <none> and password file <none>
I20260929 19:09:32.594089 13205 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:32.594166 13205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:32.594331 13205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:32.595467 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "8d98b523526443ac866c055147397f0c"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "f58f4bca6b66b5cf35b2b0e772ea03eb"
server_key_iv: "9a73517c1a538d49dd667dd30950b544"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.595837 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "8d98b523526443ac866c055147397f0c"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "f58f4bca6b66b5cf35b2b0e772ea03eb"
server_key_iv: "9a73517c1a538d49dd667dd30950b544"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.597177 13205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:32.597995 13219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.598157 13205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:32.598258 13205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "8d98b523526443ac866c055147397f0c"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "f58f4bca6b66b5cf35b2b0e772ea03eb"
server_key_iv: "9a73517c1a538d49dd667dd30950b544"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.598336 13205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:32.633157 13205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:32.633529 13205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:32.633683 13205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:32.633996 13205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:32.634354 13205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:32.634392 13205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.634415 13205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:32.634462 13205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.640197 13205 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:44943
I20260929 19:09:32.640266 13332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:44943 every 8 connection(s)
I20260929 19:09:32.640594 13205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:09:32.644887 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 13205
I20260929 19:09:32.645004 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:09:32.645223 26939 external_mini_cluster.cc:1428] Setting key dfa561e0414c9fe51f989acd58c029c1
I20260929 19:09:32.645881 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.641397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:59089 (local address 127.26.78.254:43319)
0929 19:09:32.641486 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.641488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.641694 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:09:32.642254 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.642255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.642270 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.642325 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:32.642992 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.643482 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.644417 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.644604 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.644653 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.645555 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:32.645557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:32.645557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:32.645561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:32.645581 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:32.645585 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:32.645642 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.645731 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.645757 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 19:09:32.646131 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.641357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43319 (local address 127.26.78.193:59089)
0929 19:09:32.641571 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.641607 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.642164 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.642402 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.642405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.642414 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:32.642918 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.642922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.643565 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.643566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.644334 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.644336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.644678 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.644678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.644785 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.645393 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:32.645393 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:32.645422 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:32.645444 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:32.645660 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:32.645665 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:32.645667 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.645723 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.645824 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260929 19:09:32.646252 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43319
--builtin_ntp_servers=127.26.78.212:39127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:09:32.647030 13333 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43319
I20260929 19:09:32.647130 13333 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:32.647325 13333 heartbeater.cc:511] Master 127.26.78.254:43319 requested a full tablet report, sending...
I20260929 19:09:32.647873 13145 ts_manager.cc:194] Registered new tserver with Master: 8d98b523526443ac866c055147397f0c (127.26.78.193:44943)
I20260929 19:09:32.648474 13145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.193:59089
W20260929 19:09:32.726353 13336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:32.726544 13336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:32.726567 13336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:32.727716 13336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:32.727757 13336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:32.727783 13336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:09:32.729127 13336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:39127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:32.729339 13336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:32.729614 13336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:32.734638 13341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.734671 13344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.734668 13342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:32.734882 13336 server_base.cc:1034] running on GCE node
I20260929 19:09:32.735152 13336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:32.735435 13336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:32.736598 13336 hybrid_clock.cc:648] HybridClock initialized: now 1790708972736585 us; error 31 us; skew 500 ppm
I20260929 19:09:32.737900 13336 webserver.cc:460] Webserver started at http://127.26.78.194:37141/ using document root <none> and password file <none>
I20260929 19:09:32.738137 13336 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:32.738205 13336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:32.738341 13336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:32.739383 13336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "35ee1e1e2ba744749fbbb7a79f990f09"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "cde2cacfe99fe59a10d4211bdb373a01"
server_key_iv: "fac484bee31b5f6c851f07246a9b8300"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.739737 13336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "35ee1e1e2ba744749fbbb7a79f990f09"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "cde2cacfe99fe59a10d4211bdb373a01"
server_key_iv: "fac484bee31b5f6c851f07246a9b8300"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.740996 13336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:32.741693 13350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.741900 13336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:32.742007 13336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "35ee1e1e2ba744749fbbb7a79f990f09"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "cde2cacfe99fe59a10d4211bdb373a01"
server_key_iv: "fac484bee31b5f6c851f07246a9b8300"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.742094 13336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:32.758942 13336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:32.759264 13336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:32.759418 13336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:32.759650 13336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:32.760005 13336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:32.760061 13336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.760106 13336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:32.760134 13336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.766117 13336 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:39411
I20260929 19:09:32.766189 13463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:39411 every 8 connection(s)
I20260929 19:09:32.766571 13336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:09:32.771517 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 13336
I20260929 19:09:32.771622 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:09:32.771811 26939 external_mini_cluster.cc:1428] Setting key e7c8e0e5c3b5cfb03afe0b31f11d102b
I20260929 19:09:32.772358 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.767322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:36553 (local address 127.26.78.254:43319)
0929 19:09:32.767380 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.767383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.767697 (+ 314us) server_negotiation.cc:407] Connection header received
0929 19:09:32.768256 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.768257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.768270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.768321 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:32.769009 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.769515 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.770639 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.770854 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.770943 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.771985 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:32.771986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:32.771986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:32.771992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:32.772007 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:32.772011 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:32.772088 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.772243 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.772271 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:09:32.772637 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.767341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43319 (local address 127.26.78.194:36553)
0929 19:09:32.767540 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.767602 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.768139 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.768392 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.768396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.768405 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:32.768903 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.768907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.769626 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.769627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.770537 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.770539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.770955 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.770956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.771091 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.771756 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:32.771757 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:32.771786 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:32.771809 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:32.772157 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:32.772162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:32.772164 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.772244 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.772346 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260929 19:09:32.772815 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43319
--builtin_ntp_servers=127.26.78.212:39127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:09:32.773278 13464 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43319
I20260929 19:09:32.773381 13464 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:32.773577 13464 heartbeater.cc:511] Master 127.26.78.254:43319 requested a full tablet report, sending...
I20260929 19:09:32.773968 13145 ts_manager.cc:194] Registered new tserver with Master: 35ee1e1e2ba744749fbbb7a79f990f09 (127.26.78.194:39411)
I20260929 19:09:32.774467 13145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.194:36553
W20260929 19:09:32.853446 13467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:32.853639 13467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:32.853659 13467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:32.854805 13467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:32.854852 13467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:32.854914 13467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:09:32.856225 13467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:39127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:32.856426 13467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:32.856693 13467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:32.861652 13473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.861654 13472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:32.861665 13475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:32.861804 13467 server_base.cc:1034] running on GCE node
I20260929 19:09:32.862088 13467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:32.862320 13467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:32.863479 13467 hybrid_clock.cc:648] HybridClock initialized: now 1790708972863465 us; error 25 us; skew 500 ppm
I20260929 19:09:32.864620 13467 webserver.cc:460] Webserver started at http://127.26.78.195:39349/ using document root <none> and password file <none>
I20260929 19:09:32.864872 13467 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:32.864943 13467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:32.865091 13467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:32.866250 13467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "de069646a9504ef8a5401946c50c3ef1"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "15be67f830f5b7808a638a13f08e0d4e"
server_key_iv: "7741f62b7dff84c59b13a2f9031cb434"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.866609 13467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "de069646a9504ef8a5401946c50c3ef1"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "15be67f830f5b7808a638a13f08e0d4e"
server_key_iv: "7741f62b7dff84c59b13a2f9031cb434"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.867993 13467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:32.868665 13481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.868822 13467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:32.868927 13467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "de069646a9504ef8a5401946c50c3ef1"
format_stamp: "Formatted at 2026-09-29 19:09:32 on dist-test-slave-xz51"
server_key: "15be67f830f5b7808a638a13f08e0d4e"
server_key_iv: "7741f62b7dff84c59b13a2f9031cb434"
server_key_version: "encryptionkey@0"
I20260929 19:09:32.869009 13467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:32.883935 13467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:32.884262 13467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:32.884423 13467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:32.884701 13467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:32.885061 13467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:32.885118 13467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.885152 13467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:32.885187 13467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:32.890933 13467 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:42001
I20260929 19:09:32.890975 13594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:42001 every 8 connection(s)
I20260929 19:09:32.891343 13467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:09:32.896279 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.892118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:55195 (local address 127.26.78.254:43319)
0929 19:09:32.892161 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.892163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.892412 (+ 249us) server_negotiation.cc:407] Connection header received
0929 19:09:32.892863 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.892865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.892875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.892928 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:32.893565 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.894143 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.895089 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.895265 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.895313 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.896072 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:32.896073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:32.896073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:32.896077 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:32.896091 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:32.896094 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:32.896135 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.896201 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.896222 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:09:32.896450 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.892101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43319 (local address 127.26.78.195:55195)
0929 19:09:32.892279 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.892313 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.892779 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.892983 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.892986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.892995 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:32.893489 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.893493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.894221 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.894221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.895011 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.895013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.895332 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.895333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.895436 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.895939 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:32.895939 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:32.895967 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:32.895990 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:32.896135 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:32.896138 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:32.896140 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.896186 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.896258 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260929 19:09:32.896971 13595 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43319
I20260929 19:09:32.897050 13595 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:32.897253 13595 heartbeater.cc:511] Master 127.26.78.254:43319 requested a full tablet report, sending...
I20260929 19:09:32.897632 13145 ts_manager.cc:194] Registered new tserver with Master: de069646a9504ef8a5401946c50c3ef1 (127.26.78.195:42001)
I20260929 19:09:32.898073 13145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.195:55195
I20260929 19:09:32.898306 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 13467
I20260929 19:09:32.898425 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:09:32.898643 26939 external_mini_cluster.cc:1428] Setting key 3f944dd21adf9daaa049a039daa42764
I20260929 19:09:32.899492 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:09:32.903573 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.900783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.26.78.254:43319)
0929 19:09:32.900862 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.900864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.900908 (+ 44us) server_negotiation.cc:407] Connection header received
0929 19:09:32.900957 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.900958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.900966 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.901011 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:32.901204 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.901740 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.902500 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.902671 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.902721 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.903302 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:32.903303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:32.903303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:32.903307 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:32.903319 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:32.903322 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:32.903391 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.903453 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.903462 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:09:32.905094 13145 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:09:32.906615 13145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:09:32.912738 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.908754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:44943 (local address 127.0.0.1:58554)
0929 19:09:32.909028 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.909038 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.909094 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.909375 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.909377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.909380 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.909448 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.909449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.910866 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.910867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.911658 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.911660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.912455 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.912456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.912528 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.912542 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.912614 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.912659 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":19,"spinlock_wait_cycles":384,"thread_start_us":107,"threads_started":1}
I20260929 19:09:32.912752 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.909029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.26.78.193:44943)
0929 19:09:32.909187 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.909192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.909202 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:09:32.909241 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.909243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.909260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.909346 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.909540 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.910751 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.911756 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.912374 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.912449 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.912536 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.912627 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.912664 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260929 19:09:32.913431 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.908765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39411 (local address 127.0.0.1:38376)
0929 19:09:32.909211 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.909226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.909260 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.909758 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.909758 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.909760 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.909805 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.909806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.910673 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.910674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.912035 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.912037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.913180 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.913181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.913248 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.913254 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.913306 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.913344 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":79,"thread_start_us":85,"threads_started":1}
I20260929 19:09:32.913542 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.908752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:42001 (local address 127.0.0.1:57590)
0929 19:09:32.908889 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.908929 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.909007 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.909545 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.909546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.909548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.909581 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.909582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.910845 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.910846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.912242 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.912243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.913277 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.913278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.913333 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.913336 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.913400 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.913430 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":5,"thread_start_us":75,"threads_started":1}
I20260929 19:09:32.913508 13267 tablet_service.cc:1467] Processing CreateTablet for tablet 1dbc3959ee574cbd9aa71c380059cbe7 (DEFAULT_TABLE table=test-table [id=a3e26e6eda864647b4edeed045d5edfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:32.913717 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.909224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38376 (local address 127.26.78.194:39411)
0929 19:09:32.909493 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.909497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.909507 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:09:32.909546 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.909548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.909612 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.909688 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.909909 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.910577 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.912130 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.912996 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.913059 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.913149 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.913607 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.913647 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":91,"threads_started":1}
I20260929 19:09:32.913956 13267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dbc3959ee574cbd9aa71c380059cbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:32.914152 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.909033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57590 (local address 127.26.78.195:42001)
0929 19:09:32.909384 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.909388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.909397 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:09:32.909431 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.909433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.909447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.909649 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.909698 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.910773 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.912326 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.913185 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.913257 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.913334 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.913413 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.913449 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":94,"threads_started":1}
I20260929 19:09:32.914371 13398 tablet_service.cc:1467] Processing CreateTablet for tablet 1dbc3959ee574cbd9aa71c380059cbe7 (DEFAULT_TABLE table=test-table [id=a3e26e6eda864647b4edeed045d5edfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:32.914700 13398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dbc3959ee574cbd9aa71c380059cbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:32.914920 13527 tablet_service.cc:1467] Processing CreateTablet for tablet 1dbc3959ee574cbd9aa71c380059cbe7 (DEFAULT_TABLE table=test-table [id=a3e26e6eda864647b4edeed045d5edfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:32.915232 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dbc3959ee574cbd9aa71c380059cbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:32.917207 13609 tablet_bootstrap.cc:492] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: Bootstrap starting.
I20260929 19:09:32.917791 13609 tablet_bootstrap.cc:654] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:32.918118 13609 log.cc:824] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:32.918224 13611 tablet_bootstrap.cc:492] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Bootstrap starting.
I20260929 19:09:32.918680 13610 tablet_bootstrap.cc:492] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c: Bootstrap starting.
I20260929 19:09:32.918777 13611 tablet_bootstrap.cc:654] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:32.918880 13609 tablet_bootstrap.cc:492] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: No bootstrap required, opened a new log
I20260929 19:09:32.918962 13609 ts_tablet_manager.cc:1397] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:09:32.919107 13611 log.cc:824] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:32.919291 13610 tablet_bootstrap.cc:654] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:32.919616 13610 log.cc:824] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:32.919883 13611 tablet_bootstrap.cc:492] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: No bootstrap required, opened a new log
I20260929 19:09:32.919956 13611 ts_tablet_manager.cc:1397] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:09:32.920418 13610 tablet_bootstrap.cc:492] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c: No bootstrap required, opened a new log
I20260929 19:09:32.920518 13610 ts_tablet_manager.cc:1397] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:09:32.921804 13609 raft_consensus.cc:348] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.922044 13609 raft_consensus.cc:374] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:32.921952 13611 raft_consensus.cc:348] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.922083 13609 raft_consensus.cc:729] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ee1e1e2ba744749fbbb7a79f990f09, State: Initialized, Role: FOLLOWER
I20260929 19:09:32.922103 13611 raft_consensus.cc:374] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:32.922137 13611 raft_consensus.cc:729] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de069646a9504ef8a5401946c50c3ef1, State: Initialized, Role: FOLLOWER
I20260929 19:09:32.922197 13609 consensus_queue.cc:260] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.922294 13611 consensus_queue.cc:260] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.922451 13609 ts_tablet_manager.cc:1428] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260929 19:09:32.922595 13464 heartbeater.cc:503] Master 127.26.78.254:43319 was elected leader, sending a full tablet report...
I20260929 19:09:32.922621 13595 heartbeater.cc:503] Master 127.26.78.254:43319 was elected leader, sending a full tablet report...
I20260929 19:09:32.922865 13611 ts_tablet_manager.cc:1428] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:32.923342 13610 raft_consensus.cc:348] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.923514 13610 raft_consensus.cc:374] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:32.923552 13610 raft_consensus.cc:729] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d98b523526443ac866c055147397f0c, State: Initialized, Role: FOLLOWER
I20260929 19:09:32.923678 13610 consensus_queue.cc:260] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.923959 13610 ts_tablet_manager.cc:1428] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:32.924009 13333 heartbeater.cc:503] Master 127.26.78.254:43319 was elected leader, sending a full tablet report...
I20260929 19:09:32.969667 13616 raft_consensus.cc:482] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:32.969785 13616 raft_consensus.cc:504] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.970234 13616 leader_election.cc:288] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d98b523526443ac866c055147397f0c (127.26.78.193:44943), 35ee1e1e2ba744749fbbb7a79f990f09 (127.26.78.194:39411)
I20260929 19:09:32.973207 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.970405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:44943 (local address 127.26.78.195:42333)
0929 19:09:32.970469 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.970482 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.970539 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.970709 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.970710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.970711 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.970781 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.970782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.971469 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.971470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.972279 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.972281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.972944 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.972946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.973017 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.973020 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.973095 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.973137 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 19:09:32.973207 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.970529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:42333 (local address 127.26.78.193:44943)
0929 19:09:32.970579 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.970581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.970589 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:09:32.970615 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.970615 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.970633 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.970693 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.970872 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.971383 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.972390 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.972824 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.972924 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.972933 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.973104 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.973139 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:09:32.973469 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.970498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:49167 (local address 127.26.78.194:39411)
0929 19:09:32.970547 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.970549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.970711 (+ 162us) server_negotiation.cc:407] Connection header received
0929 19:09:32.970783 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.970784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.970793 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.970879 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:32.971051 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.971613 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.972649 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.973064 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.973158 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.973165 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.973355 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.973404 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:09:32.973481 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.970405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39411 (local address 127.26.78.195:49167)
0929 19:09:32.970646 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:32.970654 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:32.970725 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:32.970880 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:32.970881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:32.970883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:32.970973 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.970974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.971703 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.971704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.972541 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:32.972544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.973160 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.973161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:32.973235 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.973238 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:32.973341 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 19:09:32.973398 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":78,"thread_start_us":92,"threads_started":1}
I20260929 19:09:32.973555 13287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dbc3959ee574cbd9aa71c380059cbe7" candidate_uuid: "de069646a9504ef8a5401946c50c3ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d98b523526443ac866c055147397f0c" is_pre_election: true
I20260929 19:09:32.973707 13287 raft_consensus.cc:2454] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de069646a9504ef8a5401946c50c3ef1 in term 0.
I20260929 19:09:32.973717 13418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dbc3959ee574cbd9aa71c380059cbe7" candidate_uuid: "de069646a9504ef8a5401946c50c3ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" is_pre_election: true
I20260929 19:09:32.973873 13418 raft_consensus.cc:2454] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de069646a9504ef8a5401946c50c3ef1 in term 0.
I20260929 19:09:32.974013 13485 leader_election.cc:302] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d98b523526443ac866c055147397f0c, de069646a9504ef8a5401946c50c3ef1; no voters:
I20260929 19:09:32.974152 13616 raft_consensus.cc:2781] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:09:32.974224 13616 raft_consensus.cc:482] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:09:32.974257 13616 raft_consensus.cc:3037] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:32.974963 13616 raft_consensus.cc:504] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.975114 13616 leader_election.cc:288] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [CANDIDATE]: Term 1 election: Requested vote from peers 8d98b523526443ac866c055147397f0c (127.26.78.193:44943), 35ee1e1e2ba744749fbbb7a79f990f09 (127.26.78.194:39411)
I20260929 19:09:32.975286 13287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dbc3959ee574cbd9aa71c380059cbe7" candidate_uuid: "de069646a9504ef8a5401946c50c3ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d98b523526443ac866c055147397f0c"
I20260929 19:09:32.975303 13418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dbc3959ee574cbd9aa71c380059cbe7" candidate_uuid: "de069646a9504ef8a5401946c50c3ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35ee1e1e2ba744749fbbb7a79f990f09"
I20260929 19:09:32.975366 13418 raft_consensus.cc:3037] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:32.975384 13287 raft_consensus.cc:3037] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:32.976076 13418 raft_consensus.cc:2454] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de069646a9504ef8a5401946c50c3ef1 in term 1.
I20260929 19:09:32.976099 13287 raft_consensus.cc:2454] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de069646a9504ef8a5401946c50c3ef1 in term 1.
I20260929 19:09:32.976266 13485 leader_election.cc:302] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d98b523526443ac866c055147397f0c, de069646a9504ef8a5401946c50c3ef1; no voters:
I20260929 19:09:32.976388 13616 raft_consensus.cc:2781] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:32.976511 13616 raft_consensus.cc:686] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 1 LEADER]: Becoming Leader. State: Replica: de069646a9504ef8a5401946c50c3ef1, State: Running, Role: LEADER
I20260929 19:09:32.976605 13616 consensus_queue.cc:237] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } }
I20260929 19:09:32.977465 13145 catalog_manager.cc:5495] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 reported cstate change: term changed from 0 to 1, leader changed from <none> to de069646a9504ef8a5401946c50c3ef1 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "de069646a9504ef8a5401946c50c3ef1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de069646a9504ef8a5401946c50c3ef1" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 42001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 } health_report { overall_health: UNKNOWN } } }
I20260929 19:09:32.985133 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:32.982841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.26.78.195:42001)
0929 19:09:32.982892 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:32.982894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:32.982903 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:09:32.982933 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:32.982934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:32.982952 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:32.983035 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:09:32.983148 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.983598 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.984352 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:32.984541 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:32.984639 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:32.984793 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:09:32.984891 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:09:32.985001 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:32.985032 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 19:09:32.985040 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:09:32.986773 13287 raft_consensus.cc:1261] T 1dbc3959ee574cbd9aa71c380059cbe7 P 8d98b523526443ac866c055147397f0c [term 1 FOLLOWER]: Refusing update from remote peer de069646a9504ef8a5401946c50c3ef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:32.986814 13418 raft_consensus.cc:1261] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 1 FOLLOWER]: Refusing update from remote peer de069646a9504ef8a5401946c50c3ef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:32.987099 13616 consensus_queue.cc:1035] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d98b523526443ac866c055147397f0c" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 44943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:32.987260 13619 consensus_queue.cc:1035] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35ee1e1e2ba744749fbbb7a79f990f09" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:32.989414 13622 mvcc.cc:205] Tried to move back new op lower bound from 7334743953351634944 to 7334743953312907264. Current Snapshot: MvccSnapshot[applied={T|T < 7334743953351634944}]
I20260929 19:09:32.990097 13623 mvcc.cc:205] Tried to move back new op lower bound from 7334743953351634944 to 7334743953312907264. Current Snapshot: MvccSnapshot[applied={T|T < 7334743953351634944}]
I20260929 19:09:32.990100 13624 mvcc.cc:205] Tried to move back new op lower bound from 7334743953351634944 to 7334743953312907264. Current Snapshot: MvccSnapshot[applied={T|T < 7334743953351634944 or (T in {7334743953351634944})}]
I20260929 19:09:32.992174 13143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:32.992264 13143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:32.993022 13143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb054358f2a14c6fa54dbf5b19583342: Sending DeleteTablet for 3 replicas of tablet 1dbc3959ee574cbd9aa71c380059cbe7
I20260929 19:09:32.993325 13527 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dbc3959ee574cbd9aa71c380059cbe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:32 UTC) from {username='slave'} at 127.0.0.1:57590
I20260929 19:09:32.993340 13398 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dbc3959ee574cbd9aa71c380059cbe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:32 UTC) from {username='slave'} at 127.0.0.1:38376
I20260929 19:09:32.993608 13632 tablet_replica.cc:330] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: stopping tablet replica
I20260929 19:09:32.993716 13632 raft_consensus.cc:2229] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:09:32.993800 13267 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dbc3959ee574cbd9aa71c380059cbe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:32 UTC) from {username='slave'} at 127.0.0.1:58554
I20260929 19:09:32.993829 13632 raft_consensus.cc:2258] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:32.994046 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 13205
I20260929 19:09:32.994289 13632 ts_tablet_manager.cc:1905] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:32.996035 13632 ts_tablet_manager.cc:1918] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:32.996129 13632 log.cc:1196] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/1dbc3959ee574cbd9aa71c380059cbe7
I20260929 19:09:32.996362 13632 ts_tablet_manager.cc:1939] T 1dbc3959ee574cbd9aa71c380059cbe7 P de069646a9504ef8a5401946c50c3ef1: Deleting consensus metadata
I20260929 19:09:32.996737 13131 catalog_manager.cc:4841] TS de069646a9504ef8a5401946c50c3ef1 (127.26.78.195:42001): tablet 1dbc3959ee574cbd9aa71c380059cbe7 (table test-table [id=a3e26e6eda864647b4edeed045d5edfa]) successfully deleted
W20260929 19:09:33.001796 13132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:44943 (error 108)
I20260929 19:09:33.001937 13633 tablet_replica.cc:330] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: stopping tablet replica
I20260929 19:09:33.002065 13633 raft_consensus.cc:2229] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:33.002188 13633 raft_consensus.cc:2258] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:33.002357 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:33.002226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:44943 (local address 127.0.0.1:58564)
0929 19:09:33.002276 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:33.002312 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 19:09:33.002475 13132 catalog_manager.cc:4584] TS 8d98b523526443ac866c055147397f0c (127.26.78.193:44943): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dbc3959ee574cbd9aa71c380059cbe7: Network error: Client connection negotiation failed: client connection to 127.26.78.193:44943: connect: Connection refused (error 111)
I20260929 19:09:33.002630 13633 ts_tablet_manager.cc:1905] T 1dbc3959ee574cbd9aa71c380059cbe7 P 35ee1e1e2ba744749fbbb7a79f990f09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:33.002799 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 13336
I20260929 19:09:33.007302 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 13467
I20260929 19:09:33.007360 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:33.007247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:39411 (local address 127.0.0.1:38384)
0929 19:09:33.007296 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:33.007328 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.194:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 19:09:33.011827 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 13114
2026-09-29T19:09:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (705 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:33 dist-test-slave-xz51 krb5kdc[13638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:33 dist-test-slave-xz51 krb5kdc[13638](info): set up 2 sockets
Sep 29 19:09:33 dist-test-slave-xz51 krb5kdc[13638](info): commencing operation
krb5kdc: starting...
W20260929 19:09:34.041965 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:34.160882 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43997
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:43997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:34.241782 13651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:34.242017 13651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:34.242040 13651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:34.243120 13651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:34.243156 13651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:34.243170 13651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:34.243182 13651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:34.243194 13651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:34.244580 13651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:43997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:34.244761 13651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:34.245014 13651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:34.249807 13656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:34.249835 13659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:34.249995 13651 server_base.cc:1034] running on GCE node
W20260929 19:09:34.250022 13657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:34.250393 13651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:34.250694 13651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:34.251842 13651 hybrid_clock.cc:648] HybridClock initialized: now 1790708974251828 us; error 30 us; skew 500 ppm
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:34.254462 13651 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:34.254922 13651 webserver.cc:460] Webserver started at http://127.26.78.254:38883/ using document root <none> and password file <none>
I20260929 19:09:34.255172 13651 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:34.255240 13651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:34.255384 13651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:34.256441 13651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "a4b72a711669432f85a1bc2885b0be13"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "c43757bed2f3a2e5f7f12841d0bcedca"
server_key_iv: "0d26c7314c7849f6a18a67349b97d973"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.256801 13651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "a4b72a711669432f85a1bc2885b0be13"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "c43757bed2f3a2e5f7f12841d0bcedca"
server_key_iv: "0d26c7314c7849f6a18a67349b97d973"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.258152 13651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:34.258927 13666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.259127 13651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:34.259236 13651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "a4b72a711669432f85a1bc2885b0be13"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "c43757bed2f3a2e5f7f12841d0bcedca"
server_key_iv: "0d26c7314c7849f6a18a67349b97d973"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.259311 13651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:34.276584 13651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:34.279099 13651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:34.279279 13651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:34.283226 13651 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:43997
I20260929 19:09:34.283263 13718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:43997 every 8 connection(s)
I20260929 19:09:34.283632 13651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:34.284535 13719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:34.285566 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 13651
I20260929 19:09:34.285696 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:34.285943 26939 external_mini_cluster.cc:1428] Setting key ee1d7d94f8d988cfdddb026bfa96c7e0
I20260929 19:09:34.287352 13719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13: Bootstrap starting.
I20260929 19:09:34.287935 13719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:34.288218 13719 log.cc:824] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:34.288964 13719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13: No bootstrap required, opened a new log
I20260929 19:09:34.291322 13719 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b72a711669432f85a1bc2885b0be13" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43997 } }
I20260929 19:09:34.291528 13719 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:34.291574 13719 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4b72a711669432f85a1bc2885b0be13, State: Initialized, Role: FOLLOWER
I20260929 19:09:34.291756 13719 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b72a711669432f85a1bc2885b0be13" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43997 } }
I20260929 19:09:34.291860 13719 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:34.291894 13719 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:34.291951 13719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:34.292688 13719 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b72a711669432f85a1bc2885b0be13" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43997 } }
I20260929 19:09:34.292814 13719 leader_election.cc:302] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4b72a711669432f85a1bc2885b0be13; no voters:
I20260929 19:09:34.293028 13719 leader_election.cc:288] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:34.293051 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 1 FOLLOWER]: Leader election won for term 1
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:34.293171 13724 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [term 1 LEADER]: Becoming Leader. State: Replica: a4b72a711669432f85a1bc2885b0be13, State: Running, Role: LEADER
I20260929 19:09:34.293246 13724 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b72a711669432f85a1bc2885b0be13" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43997 } }
I20260929 19:09:34.293309 13719 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:34.293591 13725 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4b72a711669432f85a1bc2885b0be13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b72a711669432f85a1bc2885b0be13" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43997 } } }
I20260929 19:09:34.293886 13725 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:34.293777 13726 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4b72a711669432f85a1bc2885b0be13. Latest consensus state: current_term: 1 leader_uuid: "a4b72a711669432f85a1bc2885b0be13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b72a711669432f85a1bc2885b0be13" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43997 } } }
I20260929 19:09:34.294183 13726 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:34.294214 13733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:34.294740 13733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:34.297472 13733 catalog_manager.cc:1347] Generated new cluster ID: f44d3796ef87443ca1abf9353810214f
I20260929 19:09:34.297533 13733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:34.298470 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.286478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.26.78.254:43997)
0929 19:09:34.286660 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.286670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.286704 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:09:34.287197 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.287204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.287253 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.287478 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:34.288118 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.289265 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.290375 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.290762 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.290842 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.293734 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:34.293739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:34.293739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:34.293751 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:34.296669 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.297098 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.297101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.297101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.297223 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.297459 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.297460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.297461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.297753 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:34.298026 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.298141 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.298286 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260929 19:09:34.317171 13733 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:34.317735 13733 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:34.320773 13733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13: Generated new TSK 0
I20260929 19:09:34.320986 13733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.78.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
Entry for principal kudu/127.26.78.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab.
I20260929 19:09:34.357381 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43997
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:34.454879 13745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:34.455076 13745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:34.455101 13745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:34.456234 13745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:34.456274 13745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:34.456300 13745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:09:34.457739 13745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.193.keytab
--principal=kudu/127.26.78.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:34.458041 13745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:34.458271 13745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:34.463370 13751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:34.463423 13753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:34.463580 13750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:34.463788 13745 server_base.cc:1034] running on GCE node
I20260929 19:09:34.463960 13745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:34.464169 13745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:34.465306 13745 hybrid_clock.cc:648] HybridClock initialized: now 1790708974465296 us; error 32 us; skew 500 ppm
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:34.467785 13745 init.cc:377] Logged in from keytab as kudu/127.26.78.193@KRBTEST.COM (short username kudu)
I20260929 19:09:34.468173 13745 webserver.cc:460] Webserver started at http://127.26.78.193:34961/ using document root <none> and password file <none>
I20260929 19:09:34.468364 13745 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:34.468405 13745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:34.468494 13745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:34.469494 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "4f06cde111fb43d2a5257b0c40aa6508"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "642c364789296957502ccd520fad19e2"
server_key_iv: "696e1571c372b4066ba3dc903650e6cf"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.469789 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "4f06cde111fb43d2a5257b0c40aa6508"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "642c364789296957502ccd520fad19e2"
server_key_iv: "696e1571c372b4066ba3dc903650e6cf"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.471140 13745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:34.471918 13760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.472118 13745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:34.472208 13745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "4f06cde111fb43d2a5257b0c40aa6508"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "642c364789296957502ccd520fad19e2"
server_key_iv: "696e1571c372b4066ba3dc903650e6cf"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.472280 13745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:34.490010 13745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:34.491990 13745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:34.492154 13745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:34.492424 13745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:34.492838 13745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:34.492897 13745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.492934 13745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:34.492961 13745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.498875 13745 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:42669
I20260929 19:09:34.498929 13873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:42669 every 8 connection(s)
I20260929 19:09:34.499238 13745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:09:34.503057 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 13745
I20260929 19:09:34.503168 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:09:34.503355 26939 external_mini_cluster.cc:1428] Setting key 4e061c6da303437d7a06e778258733c8
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.193@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:34.509176 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.499977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:39911 (local address 127.26.78.254:43997)
0929 19:09:34.500042 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.500045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.500257 (+ 212us) server_negotiation.cc:407] Connection header received
0929 19:09:34.500730 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.500731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.500741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.500795 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:34.501699 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.502255 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.503516 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.503781 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.503833 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.506303 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:34.506304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:34.506304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:34.506311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:34.507651 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.508181 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.508182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.508182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.508232 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.508514 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.508514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.508514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.508611 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:34.508660 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.509038 (+ 378us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.509110 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:09:34.509462 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.499965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43997 (local address 127.26.78.193:39911)
0929 19:09:34.500138 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.500174 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.500649 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.500849 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.500852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.501420 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:34.501620 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.501625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.502330 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.502331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.503376 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.503379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.503855 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.503856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.503997 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.504481 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:34.504492 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:34.506188 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:34.507739 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:34.507746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:34.507748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:34.508094 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:34.508303 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:34.508304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:34.508304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:34.508446 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:34.508680 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:34.508687 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:34.508868 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.508998 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.509179 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260929 19:09:34.510114 13874 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43997
I20260929 19:09:34.510218 13874 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:34.510421 13874 heartbeater.cc:511] Master 127.26.78.254:43997 requested a full tablet report, sending...
I20260929 19:09:34.510975 13683 ts_manager.cc:194] Registered new tserver with Master: 4f06cde111fb43d2a5257b0c40aa6508 (127.26.78.193:42669)
I20260929 19:09:34.511600 13683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.193@KRBTEST.COM'} at 127.26.78.193:39911
WARNING: no policy specified for kudu/127.26.78.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
Entry for principal kudu/127.26.78.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab.
I20260929 19:09:34.530332 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43997
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:34.613497 13879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:34.613704 13879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:34.613727 13879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:34.614933 13879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:34.614974 13879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:34.615000 13879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:09:34.616371 13879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.194.keytab
--principal=kudu/127.26.78.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:34.616551 13879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:34.616734 13879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:34.621737 13887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:34.621753 13884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:34.621773 13879 server_base.cc:1034] running on GCE node
W20260929 19:09:34.621742 13885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:34.622224 13879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:34.622434 13879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:34.623570 13879 hybrid_clock.cc:648] HybridClock initialized: now 1790708974623560 us; error 27 us; skew 500 ppm
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:34.626120 13879 init.cc:377] Logged in from keytab as kudu/127.26.78.194@KRBTEST.COM (short username kudu)
I20260929 19:09:34.626576 13879 webserver.cc:460] Webserver started at http://127.26.78.194:36205/ using document root <none> and password file <none>
I20260929 19:09:34.626765 13879 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:34.626806 13879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:34.626901 13879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:34.627884 13879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "21906987f228be0a2d0d3e07057f204b"
server_key_iv: "a982220311b145fa83586298cd908eaf"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.628180 13879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "21906987f228be0a2d0d3e07057f204b"
server_key_iv: "a982220311b145fa83586298cd908eaf"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.629405 13879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:09:34.630105 13894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.630259 13879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:34.630342 13879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "21906987f228be0a2d0d3e07057f204b"
server_key_iv: "a982220311b145fa83586298cd908eaf"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.630401 13879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:34.639500 13879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:34.641245 13879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:34.641350 13879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:34.641527 13879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:34.641774 13879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:34.641803 13879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.641875 13879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:34.641902 13879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.647455 13879 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:43391
I20260929 19:09:34.647529 14007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:43391 every 8 connection(s)
I20260929 19:09:34.647840 13879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.194@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:34.655254 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 13879
I20260929 19:09:34.655392 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:09:34.655597 26939 external_mini_cluster.cc:1428] Setting key 0bba43add80294200727142d2f550a61
I20260929 19:09:34.657262 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.648590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:47457 (local address 127.26.78.254:43997)
0929 19:09:34.648651 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.648654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.648866 (+ 212us) server_negotiation.cc:407] Connection header received
0929 19:09:34.649377 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.649378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.649390 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.649439 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:34.650478 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.650979 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.651987 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.652196 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.652282 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.654411 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:34.654412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:34.654412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:34.654419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:34.655786 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.656263 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.656264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.656265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.656333 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.656638 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.656639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.656639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.656739 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:34.656789 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.657123 (+ 334us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.657187 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:09:34.657564 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.648514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43997 (local address 127.26.78.194:47457)
0929 19:09:34.648707 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.648772 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.649266 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.649517 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.649520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.650169 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:34.650368 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.650373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.651077 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.651078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.651892 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.651894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.652291 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.652293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.652422 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.652929 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:34.652937 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:34.654322 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:34.655911 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:34.655918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:34.655920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:34.656190 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:34.656420 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:34.656421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:34.656421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:34.656572 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:34.656805 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:34.656811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:34.656988 (+ 177us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.657102 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.657286 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260929 19:09:34.658099 14008 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43997
I20260929 19:09:34.658190 14008 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:34.658377 14008 heartbeater.cc:511] Master 127.26.78.254:43997 requested a full tablet report, sending...
I20260929 19:09:34.658740 13683 ts_manager.cc:194] Registered new tserver with Master: e75e1d59d2d94b05bc43c6d6c8f335c5 (127.26.78.194:43391)
I20260929 19:09:34.659241 13683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.194@KRBTEST.COM'} at 127.26.78.194:47457
WARNING: no policy specified for kudu/127.26.78.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
Entry for principal kudu/127.26.78.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab.
I20260929 19:09:34.682566 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43997
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:34.764403 14013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:34.764608 14013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:34.764632 14013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:34.765728 14013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:34.765766 14013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:34.765792 14013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:09:34.767347 14013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.195.keytab
--principal=kudu/127.26.78.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:43997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:34.767550 14013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:34.767727 14013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:34.772859 14019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:34.772874 14018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:34.772935 14013 server_base.cc:1034] running on GCE node
W20260929 19:09:34.772862 14021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:34.773343 14013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:34.773643 14013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:34.774783 14013 hybrid_clock.cc:648] HybridClock initialized: now 1790708974774746 us; error 53 us; skew 500 ppm
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:34.777277 14013 init.cc:377] Logged in from keytab as kudu/127.26.78.195@KRBTEST.COM (short username kudu)
I20260929 19:09:34.777704 14013 webserver.cc:460] Webserver started at http://127.26.78.195:39333/ using document root <none> and password file <none>
I20260929 19:09:34.777988 14013 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:34.778036 14013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:34.778187 14013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:34.779275 14013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "d2542b8f1ad64a0da006e99f81181cc5"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "af4fb7566eff07a93fd4e1e65ab9e0d3"
server_key_iv: "d8c548bebb86b8cf4e5c19aad178dde7"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.779641 14013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "d2542b8f1ad64a0da006e99f81181cc5"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "af4fb7566eff07a93fd4e1e65ab9e0d3"
server_key_iv: "d8c548bebb86b8cf4e5c19aad178dde7"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.780943 14013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:34.781648 14028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.781875 14013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:34.781985 14013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "d2542b8f1ad64a0da006e99f81181cc5"
format_stamp: "Formatted at 2026-09-29 19:09:34 on dist-test-slave-xz51"
server_key: "af4fb7566eff07a93fd4e1e65ab9e0d3"
server_key_iv: "d8c548bebb86b8cf4e5c19aad178dde7"
server_key_version: "encryptionkey@0"
I20260929 19:09:34.782070 14013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:34.799886 14013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:34.802321 14013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:34.802493 14013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:34.802739 14013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:34.803092 14013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:34.803149 14013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.803192 14013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:34.803217 14013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:34.808975 14013 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:44361
I20260929 19:09:34.809052 14141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:44361 every 8 connection(s)
I20260929 19:09:34.809386 14013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:09:34 dist-test-slave-xz51 krb5kdc[13638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708974, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.195@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:34.817536 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14013
I20260929 19:09:34.817658 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:09:34.817864 26939 external_mini_cluster.cc:1428] Setting key 85659d7c44d52d8315fecbcc7093caf9
I20260929 19:09:34.818645 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.810200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:48247 (local address 127.26.78.254:43997)
0929 19:09:34.810263 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.810266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.810475 (+ 209us) server_negotiation.cc:407] Connection header received
0929 19:09:34.811011 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.811012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.811025 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.811082 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:34.812059 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.812571 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.813576 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.813786 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.813898 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.815901 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:34.815903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:34.815903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:34.815911 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:34.817063 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.817476 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.817477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.817477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.817538 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.817850 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.817851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.817851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.818036 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:34.818110 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.818469 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.818553 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:09:34.818864 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.810193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:43997 (local address 127.26.78.195:48247)
0929 19:09:34.810342 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.810389 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.810902 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.811160 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.811163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.811746 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:34.811947 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.811952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.812668 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.812669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.813469 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.813471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.813910 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.813911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.814034 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.814486 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:09:34.814493 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:34.815797 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:34.817149 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:34.817155 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:34.817158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:34.817397 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:34.817612 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:09:34.817613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:09:34.817614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:09:34.817745 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:09:34.818116 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:34.818123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:34.818300 (+ 177us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.818446 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.818609 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260929 19:09:34.819226 14142 heartbeater.cc:347] Connected to a master server at 127.26.78.254:43997
I20260929 19:09:34.819329 14142 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:34.819509 14142 heartbeater.cc:511] Master 127.26.78.254:43997 requested a full tablet report, sending...
I20260929 19:09:34.819854 13683 ts_manager.cc:194] Registered new tserver with Master: d2542b8f1ad64a0da006e99f81181cc5 (127.26.78.195:44361)
I20260929 19:09:34.820230 13683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.78.195@KRBTEST.COM'} at 127.26.78.195:48247
I20260929 19:09:34.829798 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:09:34.837013 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.831156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38160 (local address 127.26.78.254:43997)
0929 19:09:34.831225 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.831229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.831278 (+ 49us) server_negotiation.cc:407] Connection header received
0929 19:09:34.831335 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.831336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.831344 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.831393 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:34.831973 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.832450 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.833227 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.833410 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.833464 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.834685 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:34.834686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:34.834687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:34.834692 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:34.835927 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.836238 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.836239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.836239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.836286 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:34.836449 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:34.836450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:34.836450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:34.836586 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:34.836660 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.836761 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.836820 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:09:34.838755 13683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:09:34.840327 13683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:09:34.846800 14154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.842658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42669 (local address 127.0.0.1:47534)
0929 19:09:34.842956 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.842966 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.843025 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.843623 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.843623 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.843626 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:34.843666 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.843667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.844517 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.844518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.845797 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.845799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.846492 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.846494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.846580 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.846594 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.846663 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.846718 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":1,"thread_start_us":45,"threads_started":1}
I20260929 19:09:34.846925 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.842833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:43391 (local address 127.0.0.1:60860)
0929 19:09:34.843246 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.843256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.843314 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.843551 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.843552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.843554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:34.843593 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.843594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.844771 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.844772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.845674 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.845677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.846657 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.846658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.846723 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.846727 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.846783 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.846830 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260929 19:09:34.846979 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.843056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.26.78.194:43391)
0929 19:09:34.843237 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.843241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.843323 (+ 82us) server_negotiation.cc:407] Connection header received
0929 19:09:34.843375 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.843378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.843395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.843469 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:34.843668 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.844687 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.845756 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.846493 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.846564 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.846620 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.846817 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.846874 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260929 19:09:34.847085 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.842750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43518 (local address 127.26.78.195:44361)
0929 19:09:34.843034 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.843039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.843049 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:09:34.843090 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.843092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.843115 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.843424 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:34.843506 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.844625 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.845910 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.846569 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.846647 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.846695 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.846964 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.847007 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":96,"threads_started":1}
I20260929 19:09:34.846925 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.842657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:44361 (local address 127.0.0.1:43518)
0929 19:09:34.842819 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.842861 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.842924 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.843243 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.843247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.843250 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:34.843325 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.843327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.844733 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.844734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.845848 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.845850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.846724 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.846725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.846785 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.846788 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.846831 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.846860 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":34,"thread_start_us":84,"threads_started":1}
I20260929 19:09:34.847724 14156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.842854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.26.78.193:42669)
0929 19:09:34.843450 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.843455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.843465 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:09:34.843503 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.843505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.843522 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.843623 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:34.843729 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.844413 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.845928 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.846398 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.846491 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.846551 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.847609 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.847653 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":64,"threads_started":1}
I20260929 19:09:34.847816 14076 tablet_service.cc:1467] Processing CreateTablet for tablet 16e859603e9e4257b3899d3bafdf1dd3 (DEFAULT_TABLE table=test-table [id=225d1489dc1f4b5983a98d67e69d29ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:34.847795 13942 tablet_service.cc:1467] Processing CreateTablet for tablet 16e859603e9e4257b3899d3bafdf1dd3 (DEFAULT_TABLE table=test-table [id=225d1489dc1f4b5983a98d67e69d29ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:34.848140 13942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e859603e9e4257b3899d3bafdf1dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:34.848138 14076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e859603e9e4257b3899d3bafdf1dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:34.848276 13808 tablet_service.cc:1467] Processing CreateTablet for tablet 16e859603e9e4257b3899d3bafdf1dd3 (DEFAULT_TABLE table=test-table [id=225d1489dc1f4b5983a98d67e69d29ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:34.848513 13808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e859603e9e4257b3899d3bafdf1dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:34.851123 14159 tablet_bootstrap.cc:492] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508: Bootstrap starting.
I20260929 19:09:34.851351 14160 tablet_bootstrap.cc:492] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Bootstrap starting.
I20260929 19:09:34.851954 14159 tablet_bootstrap.cc:654] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:34.852003 14160 tablet_bootstrap.cc:654] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:34.852602 14159 log.cc:824] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:34.852602 14160 log.cc:824] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:34.852952 14161 tablet_bootstrap.cc:492] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Bootstrap starting.
I20260929 19:09:34.853454 14160 tablet_bootstrap.cc:492] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: No bootstrap required, opened a new log
I20260929 19:09:34.853539 14160 ts_tablet_manager.cc:1397] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:09:34.853547 14159 tablet_bootstrap.cc:492] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508: No bootstrap required, opened a new log
I20260929 19:09:34.853603 14161 tablet_bootstrap.cc:654] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:34.853610 14159 ts_tablet_manager.cc:1397] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:09:34.854179 14161 log.cc:824] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:34.854972 14161 tablet_bootstrap.cc:492] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: No bootstrap required, opened a new log
I20260929 19:09:34.855041 14161 ts_tablet_manager.cc:1397] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:09:34.855320 14160 raft_consensus.cc:348] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.855490 14160 raft_consensus.cc:374] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:34.855535 14160 raft_consensus.cc:729] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2542b8f1ad64a0da006e99f81181cc5, State: Initialized, Role: FOLLOWER
I20260929 19:09:34.855645 14160 consensus_queue.cc:260] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.855724 14159 raft_consensus.cc:348] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.855906 14159 raft_consensus.cc:374] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:34.855949 14159 raft_consensus.cc:729] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f06cde111fb43d2a5257b0c40aa6508, State: Initialized, Role: FOLLOWER
I20260929 19:09:34.855976 14160 ts_tablet_manager.cc:1428] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:09:34.856051 14159 consensus_queue.cc:260] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.856102 14142 heartbeater.cc:503] Master 127.26.78.254:43997 was elected leader, sending a full tablet report...
I20260929 19:09:34.856473 14159 ts_tablet_manager.cc:1428] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:34.856756 13874 heartbeater.cc:503] Master 127.26.78.254:43997 was elected leader, sending a full tablet report...
I20260929 19:09:34.857282 14161 raft_consensus.cc:348] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.857435 14161 raft_consensus.cc:374] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:34.857468 14161 raft_consensus.cc:729] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75e1d59d2d94b05bc43c6d6c8f335c5, State: Initialized, Role: FOLLOWER
I20260929 19:09:34.857542 14161 consensus_queue.cc:260] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.857719 14161 ts_tablet_manager.cc:1428] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:34.857795 14008 heartbeater.cc:503] Master 127.26.78.254:43997 was elected leader, sending a full tablet report...
I20260929 19:09:34.884061 14165 raft_consensus.cc:482] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:34.884193 14165 raft_consensus.cc:504] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.884579 14165 leader_election.cc:288] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e75e1d59d2d94b05bc43c6d6c8f335c5 (127.26.78.194:43391), 4f06cde111fb43d2a5257b0c40aa6508 (127.26.78.193:42669)
I20260929 19:09:34.887631 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.884835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:38011 (local address 127.26.78.194:43391)
0929 19:09:34.884898 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.884900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.885044 (+ 144us) server_negotiation.cc:407] Connection header received
0929 19:09:34.885117 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.885118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.885129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.885190 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:34.885374 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.885880 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.886906 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.887274 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.887331 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.887369 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.887519 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.887564 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:09:34.887629 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.884755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:43391 (local address 127.26.78.195:38011)
0929 19:09:34.884962 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.884973 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.885038 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.885201 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.885201 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.885203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:34.885291 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.885293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.885964 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.885965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.886802 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.886804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.887363 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.887364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.887443 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.887447 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.887491 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.887520 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":57,"thread_start_us":119,"threads_started":1}
I20260929 19:09:34.887686 14156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.884848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:49111 (local address 127.26.78.193:42669)
0929 19:09:34.884893 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.884895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.884940 (+ 45us) server_negotiation.cc:407] Connection header received
0929 19:09:34.884971 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.884972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.884983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.885038 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:34.885262 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.885931 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.886911 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.887287 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.887381 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.887389 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.887578 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.887612 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:09:34.887723 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.884755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42669 (local address 127.26.78.195:49111)
0929 19:09:34.884816 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.884828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:34.884889 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:34.885100 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:34.885102 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:34.885104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:34.885181 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.885183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.886016 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.886018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.886800 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:34.886802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.887393 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.887394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:34.887497 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.887501 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:34.887559 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:09:34.887595 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":29}
I20260929 19:09:34.888245 13828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e859603e9e4257b3899d3bafdf1dd3" candidate_uuid: "d2542b8f1ad64a0da006e99f81181cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f06cde111fb43d2a5257b0c40aa6508" is_pre_election: true
I20260929 19:09:34.888257 13962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e859603e9e4257b3899d3bafdf1dd3" candidate_uuid: "d2542b8f1ad64a0da006e99f81181cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" is_pre_election: true
I20260929 19:09:34.888372 13828 raft_consensus.cc:2454] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2542b8f1ad64a0da006e99f81181cc5 in term 0.
I20260929 19:09:34.888422 13962 raft_consensus.cc:2454] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2542b8f1ad64a0da006e99f81181cc5 in term 0.
I20260929 19:09:34.888609 14032 leader_election.cc:302] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f06cde111fb43d2a5257b0c40aa6508, d2542b8f1ad64a0da006e99f81181cc5; no voters:
I20260929 19:09:34.888772 14165 raft_consensus.cc:2781] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:09:34.888823 14165 raft_consensus.cc:482] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:09:34.888854 14165 raft_consensus.cc:3037] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:34.889521 14165 raft_consensus.cc:504] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.889657 14165 leader_election.cc:288] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [CANDIDATE]: Term 1 election: Requested vote from peers e75e1d59d2d94b05bc43c6d6c8f335c5 (127.26.78.194:43391), 4f06cde111fb43d2a5257b0c40aa6508 (127.26.78.193:42669)
I20260929 19:09:34.889807 13962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e859603e9e4257b3899d3bafdf1dd3" candidate_uuid: "d2542b8f1ad64a0da006e99f81181cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5"
I20260929 19:09:34.889909 13962 raft_consensus.cc:3037] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:34.889958 13828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e859603e9e4257b3899d3bafdf1dd3" candidate_uuid: "d2542b8f1ad64a0da006e99f81181cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f06cde111fb43d2a5257b0c40aa6508"
I20260929 19:09:34.890043 13828 raft_consensus.cc:3037] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:34.890745 13962 raft_consensus.cc:2454] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2542b8f1ad64a0da006e99f81181cc5 in term 1.
I20260929 19:09:34.890762 13828 raft_consensus.cc:2454] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2542b8f1ad64a0da006e99f81181cc5 in term 1.
I20260929 19:09:34.890928 14032 leader_election.cc:302] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f06cde111fb43d2a5257b0c40aa6508, d2542b8f1ad64a0da006e99f81181cc5; no voters:
I20260929 19:09:34.891067 14165 raft_consensus.cc:2781] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:34.891224 14165 raft_consensus.cc:686] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 1 LEADER]: Becoming Leader. State: Replica: d2542b8f1ad64a0da006e99f81181cc5, State: Running, Role: LEADER
I20260929 19:09:34.891295 14165 consensus_queue.cc:237] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } }
I20260929 19:09:34.892175 13683 catalog_manager.cc:5495] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2542b8f1ad64a0da006e99f81181cc5 (127.26.78.195). New cstate: current_term: 1 leader_uuid: "d2542b8f1ad64a0da006e99f81181cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2542b8f1ad64a0da006e99f81181cc5" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 } health_report { overall_health: UNKNOWN } } }
W20260929 19:09:34.898343 14009 tablet.cc:2369] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:09:34.906227 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.904069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.26.78.195:44361)
0929 19:09:34.904124 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:34.904126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:34.904134 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:09:34.904170 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:34.904170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:34.904194 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:34.904280 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:09:34.904390 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.904846 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.905532 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:34.905710 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:34.905762 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:34.905945 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:09:34.906042 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:09:34.906103 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:34.906138 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 19:09:34.906154 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:09:34.907953 13962 raft_consensus.cc:1261] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 1 FOLLOWER]: Refusing update from remote peer d2542b8f1ad64a0da006e99f81181cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:34.907980 13828 raft_consensus.cc:1261] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 1 FOLLOWER]: Refusing update from remote peer d2542b8f1ad64a0da006e99f81181cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:34.908335 14169 consensus_queue.cc:1035] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75e1d59d2d94b05bc43c6d6c8f335c5" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 43391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:34.908465 14165 consensus_queue.cc:1035] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f06cde111fb43d2a5257b0c40aa6508" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:34.910957 14173 mvcc.cc:205] Tried to move back new op lower bound from 7334743961220276224 to 7334743961155420160. Current Snapshot: MvccSnapshot[applied={T|T < 7334743961220276224}]
I20260929 19:09:34.911283 14174 mvcc.cc:205] Tried to move back new op lower bound from 7334743961220276224 to 7334743961155420160. Current Snapshot: MvccSnapshot[applied={T|T < 7334743961220276224}]
I20260929 19:09:34.913393 13683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:34.913498 13683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:34.914258 13683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4b72a711669432f85a1bc2885b0be13: Sending DeleteTablet for 3 replicas of tablet 16e859603e9e4257b3899d3bafdf1dd3
I20260929 19:09:34.914587 14076 tablet_service.cc:1514] Processing DeleteTablet for tablet 16e859603e9e4257b3899d3bafdf1dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:34 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:43518
I20260929 19:09:34.914588 13808 tablet_service.cc:1514] Processing DeleteTablet for tablet 16e859603e9e4257b3899d3bafdf1dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:34 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:47534
I20260929 19:09:34.914649 13942 tablet_service.cc:1514] Processing DeleteTablet for tablet 16e859603e9e4257b3899d3bafdf1dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:34 UTC) from {username='kudu', principal='kudu/127.26.78.254@KRBTEST.COM'} at 127.0.0.1:60860
I20260929 19:09:34.914798 14182 tablet_replica.cc:330] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508: stopping tablet replica
I20260929 19:09:34.914839 14183 tablet_replica.cc:330] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: stopping tablet replica
I20260929 19:09:34.914896 14182 raft_consensus.cc:2229] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:34.914930 14183 raft_consensus.cc:2229] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:34.914992 14182 raft_consensus.cc:2258] T 16e859603e9e4257b3899d3bafdf1dd3 P 4f06cde111fb43d2a5257b0c40aa6508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:34.915027 14183 raft_consensus.cc:2258] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:34.915294 14181 tablet_replica.cc:330] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: stopping tablet replica
I20260929 19:09:34.915426 14181 raft_consensus.cc:2229] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:09:34.915433 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 13745
I20260929 19:09:34.915465 14183 ts_tablet_manager.cc:1905] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:34.915644 14181 raft_consensus.cc:2258] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:34.916783 14181 ts_tablet_manager.cc:1905] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:34.917397 14183 ts_tablet_manager.cc:1918] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:34.917475 14183 log.cc:1196] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/16e859603e9e4257b3899d3bafdf1dd3
I20260929 19:09:34.917773 14183 ts_tablet_manager.cc:1939] T 16e859603e9e4257b3899d3bafdf1dd3 P e75e1d59d2d94b05bc43c6d6c8f335c5: Deleting consensus metadata
I20260929 19:09:34.918191 13668 catalog_manager.cc:4841] TS e75e1d59d2d94b05bc43c6d6c8f335c5 (127.26.78.194:43391): tablet 16e859603e9e4257b3899d3bafdf1dd3 (table test-table [id=225d1489dc1f4b5983a98d67e69d29ea]) successfully deleted
I20260929 19:09:34.918641 14181 ts_tablet_manager.cc:1918] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:34.918728 14181 log.cc:1196] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/16e859603e9e4257b3899d3bafdf1dd3
I20260929 19:09:34.918980 14181 ts_tablet_manager.cc:1939] T 16e859603e9e4257b3899d3bafdf1dd3 P d2542b8f1ad64a0da006e99f81181cc5: Deleting consensus metadata
I20260929 19:09:34.919453 13668 catalog_manager.cc:4841] TS d2542b8f1ad64a0da006e99f81181cc5 (127.26.78.195:44361): tablet 16e859603e9e4257b3899d3bafdf1dd3 (table test-table [id=225d1489dc1f4b5983a98d67e69d29ea]) successfully deleted
W20260929 19:09:34.923380 13670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:42669 (error 108)
I20260929 19:09:34.923789 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 13879
I20260929 19:09:34.923931 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:34.923778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42669 (local address 127.0.0.1:47542)
0929 19:09:34.923844 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:34.923894 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:42669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 19:09:34.925012 13670 catalog_manager.cc:4584] TS 4f06cde111fb43d2a5257b0c40aa6508 (127.26.78.193:42669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16e859603e9e4257b3899d3bafdf1dd3: Network error: Client connection negotiation failed: client connection to 127.26.78.193:42669: connect: Connection refused (error 111)
I20260929 19:09:34.929244 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14013
I20260929 19:09:34.933570 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 13651
2026-09-29T19:09:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1922 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T19:09:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:34Z Disabled control of system clock
I20260929 19:09:34.960731 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:45469
--webserver_interface=127.26.78.254
--webserver_port=0
--builtin_ntp_servers=127.26.78.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:09:35.040623 14188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:35.040836 14188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:35.040856 14188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:35.041947 14188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:35.041983 14188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:35.041996 14188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:35.042008 14188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:35.042021 14188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:35.043267 14188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:45469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:35.043454 14188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:35.043677 14188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:35.048592 14196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:35.048600 14194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:35.048686 14188 server_base.cc:1034] running on GCE node
W20260929 19:09:35.048616 14193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:35.049073 14188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:35.049417 14188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:35.050561 14188 hybrid_clock.cc:648] HybridClock initialized: now 1790708975050551 us; error 32 us; skew 500 ppm
I20260929 19:09:35.051783 14188 webserver.cc:460] Webserver started at http://127.26.78.254:33379/ using document root <none> and password file <none>
I20260929 19:09:35.052054 14188 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:35.052101 14188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:35.052245 14188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:35.053336 14188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "76809a46aa6247769d894b2cdab71e18"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "26fb5eda3920354043907e57954ea98e"
server_key_iv: "a3ace0e3dcd821c8301aebb39595fdfa"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.053697 14188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "76809a46aa6247769d894b2cdab71e18"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "26fb5eda3920354043907e57954ea98e"
server_key_iv: "a3ace0e3dcd821c8301aebb39595fdfa"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.055133 14188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:35.055858 14202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.056037 14188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:09:35.056137 14188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "76809a46aa6247769d894b2cdab71e18"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "26fb5eda3920354043907e57954ea98e"
server_key_iv: "a3ace0e3dcd821c8301aebb39595fdfa"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.056221 14188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:35.065786 14188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:35.066145 14188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:35.066295 14188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:35.070163 14188 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:45469
I20260929 19:09:35.070209 14254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:45469 every 8 connection(s)
I20260929 19:09:35.070572 14188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:35.071525 14255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:35.073794 14255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18: Bootstrap starting.
I20260929 19:09:35.074389 14255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:35.074627 14255 log.cc:824] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:35.075083 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14188
I20260929 19:09:35.075222 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:35.075377 14255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18: No bootstrap required, opened a new log
I20260929 19:09:35.075430 26939 external_mini_cluster.cc:1428] Setting key 0cd174f0130a1f6a69ba547dbf6483a4
I20260929 19:09:35.078100 14255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76809a46aa6247769d894b2cdab71e18" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45469 } }
I20260929 19:09:35.078291 14255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:35.078332 14255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76809a46aa6247769d894b2cdab71e18, State: Initialized, Role: FOLLOWER
I20260929 19:09:35.078460 14255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76809a46aa6247769d894b2cdab71e18" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45469 } }
I20260929 19:09:35.078537 14255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:35.078585 14255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:35.078634 14255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:35.079322 14255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76809a46aa6247769d894b2cdab71e18" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45469 } }
I20260929 19:09:35.079447 14255 leader_election.cc:302] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76809a46aa6247769d894b2cdab71e18; no voters:
I20260929 19:09:35.079646 14255 leader_election.cc:288] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:35.079679 14260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:35.079790 14260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [term 1 LEADER]: Becoming Leader. State: Replica: 76809a46aa6247769d894b2cdab71e18, State: Running, Role: LEADER
I20260929 19:09:35.079924 14255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:35.079941 14260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76809a46aa6247769d894b2cdab71e18" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45469 } }
I20260929 19:09:35.080377 14261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76809a46aa6247769d894b2cdab71e18. Latest consensus state: current_term: 1 leader_uuid: "76809a46aa6247769d894b2cdab71e18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76809a46aa6247769d894b2cdab71e18" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45469 } } }
I20260929 19:09:35.080391 14262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76809a46aa6247769d894b2cdab71e18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76809a46aa6247769d894b2cdab71e18" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 45469 } } }
I20260929 19:09:35.080511 14261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:35.080819 14262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:35.081386 14264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:35.081418 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.075965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.26.78.254:45469)
0929 19:09:35.076156 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.076163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.076187 (+ 24us) server_negotiation.cc:407] Connection header received
0929 19:09:35.076709 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.076715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.076770 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.076888 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:35.077129 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.078099 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.079236 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.079748 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.079883 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.080554 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:35.080558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:35.080558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:35.080798 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:35.080840 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:35.080855 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:35.080967 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.081039 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.081211 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260929 19:09:35.081943 14264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:35.083425 14264 catalog_manager.cc:1347] Generated new cluster ID: 1fb51e5d60444e11813359c73b233999
I20260929 19:09:35.083480 14264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:35.095230 14264 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:35.095750 14264 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:35.098713 14264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18: Generated new TSK 0
I20260929 19:09:35.098904 14264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:35.104159 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.193:0
--local_ip_for_outbound_sockets=127.26.78.193
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45469
--builtin_ntp_servers=127.26.78.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:09:35.223255 14279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:35.223551 14279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:35.223587 14279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:35.225195 14279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:35.225271 14279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:35.225311 14279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.193
I20260929 19:09:35.227322 14279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.78.193
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.193
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:35.227578 14279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:35.227944 14279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:35.233345 14287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:35.233353 14284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:35.233530 14279 server_base.cc:1034] running on GCE node
W20260929 19:09:35.233353 14285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:35.234103 14279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:35.234345 14279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:35.235512 14279 hybrid_clock.cc:648] HybridClock initialized: now 1790708975235499 us; error 30 us; skew 500 ppm
I20260929 19:09:35.236807 14279 webserver.cc:460] Webserver started at http://127.26.78.193:39579/ using document root <none> and password file <none>
I20260929 19:09:35.237063 14279 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:35.237133 14279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:35.237287 14279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:35.238652 14279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data/instance:
uuid: "677d871368a6418b853166a5a78f7c59"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "dd3f512d651fd3ab71156dadae07f411"
server_key_iv: "c59dc89acbb5aab5a81fa36c9bf0a2f6"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.239133 14279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance:
uuid: "677d871368a6418b853166a5a78f7c59"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "dd3f512d651fd3ab71156dadae07f411"
server_key_iv: "c59dc89acbb5aab5a81fa36c9bf0a2f6"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.240520 14279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:35.241348 14293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.241525 14279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:35.241622 14279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/wal
uuid: "677d871368a6418b853166a5a78f7c59"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "dd3f512d651fd3ab71156dadae07f411"
server_key_iv: "c59dc89acbb5aab5a81fa36c9bf0a2f6"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.241760 14279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:35.275813 14279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:35.276165 14279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:35.276320 14279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:35.276563 14279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:35.276928 14279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:35.276991 14279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.277035 14279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:35.277060 14279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.283125 14279 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.193:42299
I20260929 19:09:35.283212 14406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.193:42299 every 8 connection(s)
I20260929 19:09:35.283552 14279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/data/info.pb
I20260929 19:09:35.288940 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.284466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:58639 (local address 127.26.78.254:45469)
0929 19:09:35.284546 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.284549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.284763 (+ 214us) server_negotiation.cc:407] Connection header received
0929 19:09:35.285334 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.285335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.285349 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.285420 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:35.286181 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.286663 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.287645 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.287822 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.287879 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.288745 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:35.288746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:35.288746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:35.288750 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:35.288759 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:35.288762 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:35.288797 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.288858 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.288882 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 19:09:35.289139 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.284424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45469 (local address 127.26.78.193:58639)
0929 19:09:35.284641 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.284677 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.285160 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.285561 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.285565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.285577 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:35.286103 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.286107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.286789 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.286790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.287572 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.287574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.287905 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.287905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.288012 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.288619 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:35.288619 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:35.288655 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:35.288677 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:35.288800 (+ 123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:35.288802 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:35.288804 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.288855 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.288930 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260929 19:09:35.289673 14407 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45469
I20260929 19:09:35.289781 14407 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:35.290002 14407 heartbeater.cc:511] Master 127.26.78.254:45469 requested a full tablet report, sending...
I20260929 19:09:35.290498 14219 ts_manager.cc:194] Registered new tserver with Master: 677d871368a6418b853166a5a78f7c59 (127.26.78.193:42299)
I20260929 19:09:35.291105 14219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.193:58639
I20260929 19:09:35.293157 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14279
I20260929 19:09:35.293263 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-0/wal/instance
I20260929 19:09:35.293483 26939 external_mini_cluster.cc:1428] Setting key f7157b074f35f9815b3f4787842dde3b
I20260929 19:09:35.294416 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.194:0
--local_ip_for_outbound_sockets=127.26.78.194
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45469
--builtin_ntp_servers=127.26.78.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:09:35.374729 14410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:35.374972 14410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:35.375008 14410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:35.376118 14410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:35.376188 14410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:35.376230 14410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.194
I20260929 19:09:35.377559 14410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.78.194
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.194
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:35.377847 14410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:35.378144 14410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:09:35.383311 14410 server_base.cc:1034] running on GCE node
W20260929 19:09:35.383256 14418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:35.383280 14416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:35.383246 14415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:35.383673 14410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:35.383922 14410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:35.385085 14410 hybrid_clock.cc:648] HybridClock initialized: now 1790708975385077 us; error 40 us; skew 500 ppm
I20260929 19:09:35.386441 14410 webserver.cc:460] Webserver started at http://127.26.78.194:44817/ using document root <none> and password file <none>
I20260929 19:09:35.386686 14410 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:35.386758 14410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:35.386895 14410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:35.387931 14410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data/instance:
uuid: "d6be85a1da934b7ea59aedd86dbcdf65"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "eaaf8df9d498564d0eb9fe8dea350fe1"
server_key_iv: "1c9f98131299330d2226352d8c3e40c8"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.388285 14410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance:
uuid: "d6be85a1da934b7ea59aedd86dbcdf65"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "eaaf8df9d498564d0eb9fe8dea350fe1"
server_key_iv: "1c9f98131299330d2226352d8c3e40c8"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.389644 14410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:35.390406 14424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.390581 14410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.390676 14410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal
uuid: "d6be85a1da934b7ea59aedd86dbcdf65"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "eaaf8df9d498564d0eb9fe8dea350fe1"
server_key_iv: "1c9f98131299330d2226352d8c3e40c8"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.390761 14410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:35.410104 14410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:35.410426 14410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:35.410578 14410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:35.410817 14410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:35.411139 14410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:35.411195 14410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.411235 14410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:35.411264 14410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.416893 14410 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.194:40409
I20260929 19:09:35.416944 14537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.194:40409 every 8 connection(s)
I20260929 19:09:35.417266 14410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/data/info.pb
I20260929 19:09:35.419467 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14410
I20260929 19:09:35.419605 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal/instance
I20260929 19:09:35.419808 26939 external_mini_cluster.cc:1428] Setting key c085a7d3feb27c672493d4a7c01f25cb
I20260929 19:09:35.420817 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.78.195:0
--local_ip_for_outbound_sockets=127.26.78.195
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45469
--builtin_ntp_servers=127.26.78.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:09:35.423019 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.417979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.194:59239 (local address 127.26.78.254:45469)
0929 19:09:35.418049 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.418050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.418287 (+ 237us) server_negotiation.cc:407] Connection header received
0929 19:09:35.418740 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.418741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.418751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.418804 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:35.419537 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.420222 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.421456 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.421665 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.421744 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.422740 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:35.422741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:35.422741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:35.422746 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:35.422761 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:35.422765 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:35.422811 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.422937 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.422959 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:09:35.423273 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.417969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45469 (local address 127.26.78.194:59239)
0929 19:09:35.418147 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.418182 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.418653 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.418865 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.418868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.418876 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:35.419441 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.419448 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.420314 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.420315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.421357 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.421359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.421763 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.421764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.421967 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.422574 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:35.422575 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:35.422616 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:35.422648 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:35.422834 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:35.422839 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:35.422842 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.422929 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.423036 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260929 19:09:35.423763 14538 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45469
I20260929 19:09:35.423856 14538 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:35.424058 14538 heartbeater.cc:511] Master 127.26.78.254:45469 requested a full tablet report, sending...
I20260929 19:09:35.424449 14219 ts_manager.cc:194] Registered new tserver with Master: d6be85a1da934b7ea59aedd86dbcdf65 (127.26.78.194:40409)
I20260929 19:09:35.424823 14219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.194:59239
W20260929 19:09:35.504467 14541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:35.504704 14541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:35.504740 14541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:35.505976 14541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:35.506031 14541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:35.506071 14541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.78.195
I20260929 19:09:35.507395 14541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.78.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.78.195
--webserver_port=0
--tserver_master_addrs=127.26.78.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.78.195
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:35.507660 14541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:35.507958 14541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:09:35.513197 14541 server_base.cc:1034] running on GCE node
W20260929 19:09:35.513166 14546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:35.513198 14547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:35.513144 14549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:35.513610 14541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:35.513902 14541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:35.515071 14541 hybrid_clock.cc:648] HybridClock initialized: now 1790708975515060 us; error 25 us; skew 500 ppm
I20260929 19:09:35.516402 14541 webserver.cc:460] Webserver started at http://127.26.78.195:43777/ using document root <none> and password file <none>
I20260929 19:09:35.516667 14541 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:35.516737 14541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:35.516875 14541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:35.518015 14541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data/instance:
uuid: "371301b037124255bf55b68d17de6966"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "40d6f3b5693f788879248760802eb67e"
server_key_iv: "36895f44165efdf7de6cf3ee13dcba46"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.518375 14541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance:
uuid: "371301b037124255bf55b68d17de6966"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "40d6f3b5693f788879248760802eb67e"
server_key_iv: "36895f44165efdf7de6cf3ee13dcba46"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.519716 14541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:09:35.520476 14555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.520650 14541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:35.520745 14541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal
uuid: "371301b037124255bf55b68d17de6966"
format_stamp: "Formatted at 2026-09-29 19:09:35 on dist-test-slave-xz51"
server_key: "40d6f3b5693f788879248760802eb67e"
server_key_iv: "36895f44165efdf7de6cf3ee13dcba46"
server_key_version: "encryptionkey@0"
I20260929 19:09:35.520828 14541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:35.528378 14541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:35.528661 14541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:35.528806 14541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:35.529021 14541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:09:35.529330 14541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:09:35.529386 14541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.529426 14541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:09:35.529452 14541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:35.535351 14541 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.195:40847
I20260929 19:09:35.535390 14668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.195:40847 every 8 connection(s)
I20260929 19:09:35.535718 14541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/data/info.pb
I20260929 19:09:35.536258 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14541
I20260929 19:09:35.536381 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal/instance
I20260929 19:09:35.536621 26939 external_mini_cluster.cc:1428] Setting key 6afcd99f431552a2530ead4aaa049c54
I20260929 19:09:35.541018 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.536422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.195:59933 (local address 127.26.78.254:45469)
0929 19:09:35.536482 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.536484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.537032 (+ 548us) server_negotiation.cc:407] Connection header received
0929 19:09:35.537612 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.537613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.537624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.537689 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:35.538409 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.538885 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.539819 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.540006 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.540059 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.540792 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:35.540793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:35.540793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:35.540797 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:35.540811 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:35.540815 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:35.540857 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.540938 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.540959 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:09:35.541246 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.536445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:45469 (local address 127.26.78.195:59933)
0929 19:09:35.536850 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.536895 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.537503 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.537746 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.537749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.537758 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:09:35.538315 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.538320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.538963 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.538964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.539746 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.539748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.540077 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.540078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.540180 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.540668 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:09:35.540668 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:09:35.540695 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:09:35.540716 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:09:35.540870 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:09:35.540874 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:09:35.540876 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.540928 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.541001 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":226,"threads_started":1}
I20260929 19:09:35.541709 14669 heartbeater.cc:347] Connected to a master server at 127.26.78.254:45469
I20260929 19:09:35.541783 14669 heartbeater.cc:464] Registering TS with master...
I20260929 19:09:35.542009 14669 heartbeater.cc:511] Master 127.26.78.254:45469 requested a full tablet report, sending...
I20260929 19:09:35.542369 14219 ts_manager.cc:194] Registered new tserver with Master: 371301b037124255bf55b68d17de6966 (127.26.78.195:40847)
I20260929 19:09:35.542729 14219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.78.195:59933
I20260929 19:09:35.548372 26939 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:09:35.552368 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.549717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40530 (local address 127.26.78.254:45469)
0929 19:09:35.549774 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.549776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.549917 (+ 141us) server_negotiation.cc:407] Connection header received
0929 19:09:35.549961 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.549961 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.549970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.550021 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:35.550151 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.550616 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.551298 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.551488 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.551537 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.552189 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:35.552190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:35.552190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:35.552195 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:35.552206 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:35.552209 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:35.552253 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.552286 (+ 33us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.552303 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:09:35.554014 14219 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:09:35.555495 14219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:09:35.561877 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.557735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.26.78.194:40409)
0929 19:09:35.557952 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.557957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.558304 (+ 347us) server_negotiation.cc:407] Connection header received
0929 19:09:35.558365 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.558367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.558382 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.558454 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:35.558618 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.559334 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.560791 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.561447 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.561541 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.561632 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.561733 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.561770 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":145,"threads_started":1}
I20260929 19:09:35.562065 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.557556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:40409 (local address 127.0.0.1:52328)
0929 19:09:35.558196 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.558207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.558250 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.558520 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.558521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.558522 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:35.558562 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.558563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.559430 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.559431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.560701 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.560703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.561545 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.561546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.561620 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.561633 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.561704 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.561766 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":7,"thread_start_us":65,"threads_started":1}
I20260929 19:09:35.562258 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.557700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.26.78.195:40847)
0929 19:09:35.558057 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.558062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.558073 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:09:35.558166 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.558169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.558185 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.558262 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:35.558498 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.559648 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.560878 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.561479 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.561548 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.561630 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.562038 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.562075 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":80,"threads_started":1}
I20260929 19:09:35.562381 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.557556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:40847 (local address 127.0.0.1:55144)
0929 19:09:35.557980 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.558024 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.558091 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.558337 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.558340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.558342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:35.558418 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.558420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.559734 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.559735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.560789 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.560790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.561665 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.561666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.561725 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.561728 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.561780 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.561821 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":67,"threads_started":1}
I20260929 19:09:35.562646 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.557584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42299 (local address 127.0.0.1:42664)
0929 19:09:35.558654 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.558663 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.558706 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.558990 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.558991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.558992 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:35.559025 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.559026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.560203 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.560204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.561118 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.561121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.562394 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.562395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.562461 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.562465 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.562541 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.562582 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260929 19:09:35.562940 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.557849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.26.78.193:42299)
0929 19:09:35.558016 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.558019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.558780 (+ 761us) server_negotiation.cc:407] Connection header received
0929 19:09:35.558825 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.558827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.558842 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.558906 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:35.559089 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.560117 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.561210 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.562104 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.562244 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.562772 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.562801 (+ 29us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.562866 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260929 19:09:35.562834 14472 tablet_service.cc:1467] Processing CreateTablet for tablet 868b6fc053dd404c9e83b3bf8a83c9ec (DEFAULT_TABLE table=test-table [id=f659d77dc75743feb3c0008c76b17a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:35.563164 14603 tablet_service.cc:1467] Processing CreateTablet for tablet 868b6fc053dd404c9e83b3bf8a83c9ec (DEFAULT_TABLE table=test-table [id=f659d77dc75743feb3c0008c76b17a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:35.563239 14472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868b6fc053dd404c9e83b3bf8a83c9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:35.563410 14603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868b6fc053dd404c9e83b3bf8a83c9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:35.563612 14341 tablet_service.cc:1467] Processing CreateTablet for tablet 868b6fc053dd404c9e83b3bf8a83c9ec (DEFAULT_TABLE table=test-table [id=f659d77dc75743feb3c0008c76b17a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:09:35.563915 14341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868b6fc053dd404c9e83b3bf8a83c9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:35.565759 14683 tablet_bootstrap.cc:492] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Bootstrap starting.
I20260929 19:09:35.566370 14683 tablet_bootstrap.cc:654] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:35.566634 14684 tablet_bootstrap.cc:492] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Bootstrap starting.
I20260929 19:09:35.566685 14683 log.cc:824] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:35.567265 14684 tablet_bootstrap.cc:654] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:35.567402 14683 tablet_bootstrap.cc:492] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: No bootstrap required, opened a new log
I20260929 19:09:35.567478 14683 ts_tablet_manager.cc:1397] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:09:35.567615 14684 log.cc:824] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:35.567885 14686 tablet_bootstrap.cc:492] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59: Bootstrap starting.
I20260929 19:09:35.568308 14684 tablet_bootstrap.cc:492] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: No bootstrap required, opened a new log
I20260929 19:09:35.568395 14684 ts_tablet_manager.cc:1397] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:09:35.568564 14686 tablet_bootstrap.cc:654] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:35.568859 14686 log.cc:824] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:35.570034 14686 tablet_bootstrap.cc:492] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59: No bootstrap required, opened a new log
I20260929 19:09:35.570114 14686 ts_tablet_manager.cc:1397] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:09:35.570192 14683 raft_consensus.cc:348] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.570358 14683 raft_consensus.cc:374] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:35.570393 14683 raft_consensus.cc:729] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 371301b037124255bf55b68d17de6966, State: Initialized, Role: FOLLOWER
I20260929 19:09:35.570538 14683 consensus_queue.cc:260] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.570693 14684 raft_consensus.cc:348] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.570859 14684 raft_consensus.cc:374] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:35.570854 14683 ts_tablet_manager.cc:1428] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 19:09:35.570905 14684 raft_consensus.cc:729] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6be85a1da934b7ea59aedd86dbcdf65, State: Initialized, Role: FOLLOWER
I20260929 19:09:35.570923 14669 heartbeater.cc:503] Master 127.26.78.254:45469 was elected leader, sending a full tablet report...
I20260929 19:09:35.571017 14684 consensus_queue.cc:260] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.571256 14684 ts_tablet_manager.cc:1428] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:35.571309 14538 heartbeater.cc:503] Master 127.26.78.254:45469 was elected leader, sending a full tablet report...
I20260929 19:09:35.572607 14686 raft_consensus.cc:348] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.572729 14686 raft_consensus.cc:374] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:35.572750 14686 raft_consensus.cc:729] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677d871368a6418b853166a5a78f7c59, State: Initialized, Role: FOLLOWER
I20260929 19:09:35.572854 14686 consensus_queue.cc:260] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.573138 14686 ts_tablet_manager.cc:1428] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:09:35.573182 14407 heartbeater.cc:503] Master 127.26.78.254:45469 was elected leader, sending a full tablet report...
W20260929 19:09:35.667798 14539 tablet.cc:2369] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:09:35.763608 14691 raft_consensus.cc:482] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:09:35.763782 14691 raft_consensus.cc:504] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.764249 14691 leader_election.cc:288] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 371301b037124255bf55b68d17de6966 (127.26.78.195:40847), d6be85a1da934b7ea59aedd86dbcdf65 (127.26.78.194:40409)
I20260929 19:09:35.767558 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.764543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:56877 (local address 127.26.78.194:40409)
0929 19:09:35.764583 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.764585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.764631 (+ 46us) server_negotiation.cc:407] Connection header received
0929 19:09:35.764701 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.764702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.764711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.764767 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:35.764975 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.765782 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.766829 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.767181 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.767237 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.767269 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.767433 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.767483 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:09:35.767561 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.764422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.195:40847 (local address 127.26.78.193:37861)
0929 19:09:35.764495 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.764507 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.764559 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.764740 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.764741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.764759 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:35.764843 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.764845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.765866 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.765867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.766689 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.766691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.767297 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.767298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.767371 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.767374 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.767449 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.767494 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":28}
I20260929 19:09:35.767570 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.764532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.78.193:37861 (local address 127.26.78.195:40847)
0929 19:09:35.764581 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.764584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.764611 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:09:35.764643 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.764644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.764654 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.764709 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:09:35.764975 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.765734 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.766787 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.767186 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.767280 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.767289 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.767466 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.767509 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:09:35.767570 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.764456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.194:40409 (local address 127.26.78.193:56877)
0929 19:09:35.764580 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.764589 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:09:35.764638 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:09:35.764779 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:09:35.764780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:09:35.764781 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:09:35.764845 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.764846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.765895 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.765897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.766718 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:09:35.766721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.767273 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.767274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:09:35.767352 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.767358 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:09:35.767406 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 19:09:35.767467 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260929 19:09:35.767920 14623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868b6fc053dd404c9e83b3bf8a83c9ec" candidate_uuid: "677d871368a6418b853166a5a78f7c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "371301b037124255bf55b68d17de6966" is_pre_election: true
I20260929 19:09:35.768000 14492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868b6fc053dd404c9e83b3bf8a83c9ec" candidate_uuid: "677d871368a6418b853166a5a78f7c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" is_pre_election: true
I20260929 19:09:35.768079 14623 raft_consensus.cc:2454] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677d871368a6418b853166a5a78f7c59 in term 0.
I20260929 19:09:35.768177 14492 raft_consensus.cc:2454] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677d871368a6418b853166a5a78f7c59 in term 0.
I20260929 19:09:35.768303 14294 leader_election.cc:302] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 371301b037124255bf55b68d17de6966, 677d871368a6418b853166a5a78f7c59; no voters:
I20260929 19:09:35.768469 14691 raft_consensus.cc:2781] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:09:35.768553 14691 raft_consensus.cc:482] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:09:35.768589 14691 raft_consensus.cc:3037] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:35.769312 14691 raft_consensus.cc:504] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.769464 14691 leader_election.cc:288] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [CANDIDATE]: Term 1 election: Requested vote from peers 371301b037124255bf55b68d17de6966 (127.26.78.195:40847), d6be85a1da934b7ea59aedd86dbcdf65 (127.26.78.194:40409)
I20260929 19:09:35.769603 14492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868b6fc053dd404c9e83b3bf8a83c9ec" candidate_uuid: "677d871368a6418b853166a5a78f7c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6be85a1da934b7ea59aedd86dbcdf65"
I20260929 19:09:35.769639 14623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868b6fc053dd404c9e83b3bf8a83c9ec" candidate_uuid: "677d871368a6418b853166a5a78f7c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "371301b037124255bf55b68d17de6966"
I20260929 19:09:35.769699 14492 raft_consensus.cc:3037] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:35.769790 14623 raft_consensus.cc:3037] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:35.770471 14492 raft_consensus.cc:2454] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677d871368a6418b853166a5a78f7c59 in term 1.
I20260929 19:09:35.770642 14623 raft_consensus.cc:2454] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677d871368a6418b853166a5a78f7c59 in term 1.
I20260929 19:09:35.770663 14296 leader_election.cc:302] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 677d871368a6418b853166a5a78f7c59, d6be85a1da934b7ea59aedd86dbcdf65; no voters:
I20260929 19:09:35.770737 14691 raft_consensus.cc:2781] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:35.770902 14691 raft_consensus.cc:686] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [term 1 LEADER]: Becoming Leader. State: Replica: 677d871368a6418b853166a5a78f7c59, State: Running, Role: LEADER
I20260929 19:09:35.771010 14691 consensus_queue.cc:237] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } }
I20260929 19:09:35.771855 14219 catalog_manager.cc:5495] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 677d871368a6418b853166a5a78f7c59 (127.26.78.193). New cstate: current_term: 1 leader_uuid: "677d871368a6418b853166a5a78f7c59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "677d871368a6418b853166a5a78f7c59" member_type: VOTER last_known_addr { host: "127.26.78.193" port: 42299 } health_report { overall_health: HEALTHY } } }
W20260929 19:09:35.784281 14408 tablet.cc:2369] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:09:35.786769 14670 tablet.cc:2369] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:09:35.791304 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.788902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42672 (local address 127.26.78.193:42299)
0929 19:09:35.788963 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:35.788966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:35.788988 (+ 22us) server_negotiation.cc:407] Connection header received
0929 19:09:35.789024 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:35.789025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:35.789045 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:35.789127 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:09:35.789262 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.789746 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.790525 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:35.790736 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:35.790813 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:35.790965 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:09:35.791072 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:09:35.791167 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:35.791192 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 19:09:35.791212 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:09:35.793187 14623 raft_consensus.cc:1261] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 1 FOLLOWER]: Refusing update from remote peer 677d871368a6418b853166a5a78f7c59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:35.793188 14492 raft_consensus.cc:1261] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 1 FOLLOWER]: Refusing update from remote peer 677d871368a6418b853166a5a78f7c59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:09:35.793576 14693 consensus_queue.cc:1035] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "371301b037124255bf55b68d17de6966" member_type: VOTER last_known_addr { host: "127.26.78.195" port: 40847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:35.793789 14691 consensus_queue.cc:1035] T 868b6fc053dd404c9e83b3bf8a83c9ec P 677d871368a6418b853166a5a78f7c59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6be85a1da934b7ea59aedd86dbcdf65" member_type: VOTER last_known_addr { host: "127.26.78.194" port: 40409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:09:35.796206 14696 mvcc.cc:205] Tried to move back new op lower bound from 7334743964845735936 to 7334743964758704128. Current Snapshot: MvccSnapshot[applied={T|T < 7334743964845735936}]
I20260929 19:09:35.796988 14698 mvcc.cc:205] Tried to move back new op lower bound from 7334743964845735936 to 7334743964758704128. Current Snapshot: MvccSnapshot[applied={T|T < 7334743964845735936}]
I20260929 19:09:35.798805 14219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:35.798902 14219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:09:35.799607 14219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76809a46aa6247769d894b2cdab71e18: Sending DeleteTablet for 3 replicas of tablet 868b6fc053dd404c9e83b3bf8a83c9ec
I20260929 19:09:35.799921 14603 tablet_service.cc:1514] Processing DeleteTablet for tablet 868b6fc053dd404c9e83b3bf8a83c9ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:35 UTC) from {username='slave'} at 127.0.0.1:55144
I20260929 19:09:35.800148 14705 tablet_replica.cc:330] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: stopping tablet replica
I20260929 19:09:35.800184 14341 tablet_service.cc:1514] Processing DeleteTablet for tablet 868b6fc053dd404c9e83b3bf8a83c9ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:35 UTC) from {username='slave'} at 127.0.0.1:42664
I20260929 19:09:35.800256 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14279
I20260929 19:09:35.800386 14705 raft_consensus.cc:2229] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:35.800535 14705 raft_consensus.cc:2258] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:35.800992 14705 ts_tablet_manager.cc:1905] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:35.802614 14705 ts_tablet_manager.cc:1918] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:35.802686 14705 log.cc:1196] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-2/wal/wals/868b6fc053dd404c9e83b3bf8a83c9ec
I20260929 19:09:35.802947 14705 ts_tablet_manager.cc:1939] T 868b6fc053dd404c9e83b3bf8a83c9ec P 371301b037124255bf55b68d17de6966: Deleting consensus metadata
I20260929 19:09:35.803112 14472 tablet_service.cc:1514] Processing DeleteTablet for tablet 868b6fc053dd404c9e83b3bf8a83c9ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:09:35 UTC) from {username='slave'} at 127.0.0.1:52328
I20260929 19:09:35.803339 14707 tablet_replica.cc:330] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: stopping tablet replica
I20260929 19:09:35.803368 14203 catalog_manager.cc:4841] TS 371301b037124255bf55b68d17de6966 (127.26.78.195:40847): tablet 868b6fc053dd404c9e83b3bf8a83c9ec (table test-table [id=f659d77dc75743feb3c0008c76b17a13]) successfully deleted
I20260929 19:09:35.803443 14707 raft_consensus.cc:2229] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:09:35.803544 14707 raft_consensus.cc:2258] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:09:35.803941 14707 ts_tablet_manager.cc:1905] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:09:35.805147 14707 ts_tablet_manager.cc:1918] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:09:35.805203 14707 log.cc:1196] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Deleting WAL directory at /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790708710206825-26939-0/minicluster-data/ts-1/wal/wals/868b6fc053dd404c9e83b3bf8a83c9ec
I20260929 19:09:35.805436 14707 ts_tablet_manager.cc:1939] T 868b6fc053dd404c9e83b3bf8a83c9ec P d6be85a1da934b7ea59aedd86dbcdf65: Deleting consensus metadata
I20260929 19:09:35.805864 14205 catalog_manager.cc:4841] TS d6be85a1da934b7ea59aedd86dbcdf65 (127.26.78.194:40409): tablet 868b6fc053dd404c9e83b3bf8a83c9ec (table test-table [id=f659d77dc75743feb3c0008c76b17a13]) successfully deleted
W20260929 19:09:35.806701 14206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.78.193:42299 (error 108)
I20260929 19:09:35.807183 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:35.807056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.193:42299 (local address 127.0.0.1:42686)
0929 19:09:35.807110 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 19:09:35.807149 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.193:42299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 19:09:35.807188 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14410
W20260929 19:09:35.807292 14206 catalog_manager.cc:4584] TS 677d871368a6418b853166a5a78f7c59 (127.26.78.193:42299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 868b6fc053dd404c9e83b3bf8a83c9ec: Network error: Client connection negotiation failed: client connection to 127.26.78.193:42299: connect: Connection refused (error 111)
I20260929 19:09:35.812042 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14541
I20260929 19:09:35.816212 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14188
2026-09-29T19:09:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (881 ms)
[----------] 6 tests from EncryptionPolicyTest (8289 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:35 dist-test-slave-xz51 krb5kdc[14711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:35 dist-test-slave-xz51 krb5kdc[14711](info): set up 2 sockets
Sep 29 19:09:35 dist-test-slave-xz51 krb5kdc[14711](info): commencing operation
krb5kdc: starting...
W20260929 19:09:36.853865 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:36 dist-test-slave-xz51 krb5kdc[14711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:36.974975 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:37.055104 14724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:37.055354 14724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:37.055394 14724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:37.056527 14724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:37.056609 14724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:37.056632 14724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:37.056653 14724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:37.056682 14724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:37.058234 14724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43469
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:37.058543 14724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:37.058786 14724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:37.063685 14732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:37.063695 14730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:37.063704 14729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:37.063943 14724 server_base.cc:1034] running on GCE node
I20260929 19:09:37.064268 14724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:37.064574 14724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:37.065719 14724 hybrid_clock.cc:648] HybridClock initialized: now 1790708977065699 us; error 41 us; skew 500 ppm
Sep 29 19:09:37 dist-test-slave-xz51 krb5kdc[14711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708977, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:37.068399 14724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:37.068941 14724 webserver.cc:460] Webserver started at http://127.0.0.1:46687/ using document root <none> and password file <none>
I20260929 19:09:37.069172 14724 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:37.069248 14724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:37.069406 14724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:37.070571 14724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "78eeb82903754716b5ea182b809ef58a"
format_stamp: "Formatted at 2026-09-29 19:09:37 on dist-test-slave-xz51"
server_key: "93e9ead8b3738c73cabbae41f861d60b"
server_key_iv: "32ed56a0963c0d03683c1a01b803037c"
server_key_version: "encryptionkey@0"
I20260929 19:09:37.070947 14724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "78eeb82903754716b5ea182b809ef58a"
format_stamp: "Formatted at 2026-09-29 19:09:37 on dist-test-slave-xz51"
server_key: "93e9ead8b3738c73cabbae41f861d60b"
server_key_iv: "32ed56a0963c0d03683c1a01b803037c"
server_key_version: "encryptionkey@0"
I20260929 19:09:37.072286 14724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:37.073064 14739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:37.073235 14724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:37.073338 14724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "78eeb82903754716b5ea182b809ef58a"
format_stamp: "Formatted at 2026-09-29 19:09:37 on dist-test-slave-xz51"
server_key: "93e9ead8b3738c73cabbae41f861d60b"
server_key_iv: "32ed56a0963c0d03683c1a01b803037c"
server_key_version: "encryptionkey@0"
I20260929 19:09:37.073423 14724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:37.081944 14724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:37.084228 14724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:37.084389 14724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:37.088356 14724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43469
I20260929 19:09:37.088393 14791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43469 every 8 connection(s)
I20260929 19:09:37.088763 14724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:37.089591 14792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:37.089985 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14724
I20260929 19:09:37.090101 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:37.090312 26939 external_mini_cluster.cc:1428] Setting key b9c3c0f29959a659e091846bd24bfc21
I20260929 19:09:37.092741 14792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a: Bootstrap starting.
I20260929 19:09:37.093330 14792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:37.093616 14792 log.cc:824] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:37.094303 14792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a: No bootstrap required, opened a new log
I20260929 19:09:37.096076 14792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eeb82903754716b5ea182b809ef58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260929 19:09:37.096232 14792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:37.096282 14792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eeb82903754716b5ea182b809ef58a, State: Initialized, Role: FOLLOWER
I20260929 19:09:37.096412 14792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eeb82903754716b5ea182b809ef58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260929 19:09:37.096477 14792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:37.096498 14792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:37.096558 14792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 0 FOLLOWER]: Advancing to term 1
Sep 29 19:09:37 dist-test-slave-xz51 krb5kdc[14711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:09:37.097131 14792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eeb82903754716b5ea182b809ef58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260929 19:09:37.097249 14792 leader_election.cc:302] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78eeb82903754716b5ea182b809ef58a; no voters:
I20260929 19:09:37.097460 14797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:37.097430 14792 leader_election.cc:288] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:37.097631 14797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [term 1 LEADER]: Becoming Leader. State: Replica: 78eeb82903754716b5ea182b809ef58a, State: Running, Role: LEADER
I20260929 19:09:37.097723 14797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eeb82903754716b5ea182b809ef58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260929 19:09:37.097767 14792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:37.098066 14798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78eeb82903754716b5ea182b809ef58a. Latest consensus state: current_term: 1 leader_uuid: "78eeb82903754716b5ea182b809ef58a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eeb82903754716b5ea182b809ef58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } } }
I20260929 19:09:37.098196 14798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:37.098083 14797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78eeb82903754716b5ea182b809ef58a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eeb82903754716b5ea182b809ef58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } } }
I20260929 19:09:37.098328 14797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:37.098609 14805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:37.099102 14805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:37.100692 14805 catalog_manager.cc:1347] Generated new cluster ID: faac7c37e71f4e93873929fe625e3d6e
I20260929 19:09:37.100747 14805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:37.101531 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:37.090891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52097 (local address 127.0.0.1:43469)
0929 19:09:37.091084 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:37.091091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:37.091115 (+ 24us) server_negotiation.cc:407] Connection header received
0929 19:09:37.091529 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:37.091535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:37.091581 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:37.091696 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:37.092508 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:37.093440 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:37.094305 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:37.094764 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:37.094880 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:37.097396 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:37.097400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:37.097401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:37.097412 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:37.099624 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:37.100036 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:37.100039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:37.100039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:37.100144 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:37.100391 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:37.100391 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:37.100392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:37.100681 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:37.100846 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:37.101249 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 19:09:37.101341 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260929 19:09:37.102461 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:37.113715 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:37.106284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38983 (local address 127.0.0.1:43469)
0929 19:09:37.106320 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:37.106321 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:37.106371 (+ 50us) server_negotiation.cc:407] Connection header received
0929 19:09:37.106449 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:37.106450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:37.106458 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:37.106509 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:37.107188 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:37.107921 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:37.108975 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:37.109299 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:37.109362 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:37.110854 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:37.110855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:37.110856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:37.110862 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:37.112470 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:37.112900 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:37.112901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:37.112901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:37.112956 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:37.113189 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:37.113189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:37.113190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:37.113312 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:37.113404 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:37.113560 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 19:09:37.113617 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":16}
I20260929 19:09:37.128247 14805 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:37.129009 14805 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:37.134176 14805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78eeb82903754716b5ea182b809ef58a: Generated new TSK 0
I20260929 19:09:37.134403 14805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:37.142843 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14724
W20260929 19:09:37.147392 14709 connection.cc:535] client connection to 127.0.0.1:43469 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43469): Connection reset by peer (error 104)
2026-09-29T19:09:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:37 dist-test-slave-xz51 krb5kdc[14823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:37 dist-test-slave-xz51 krb5kdc[14823](info): set up 2 sockets
Sep 29 19:09:37 dist-test-slave-xz51 krb5kdc[14823](info): commencing operation
krb5kdc: starting...
W20260929 19:09:38.182138 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:38 dist-test-slave-xz51 krb5kdc[14823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:38.300854 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:38.383623 14836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:38.383841 14836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:38.383865 14836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:38.385015 14836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:38.385097 14836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:38.385126 14836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:38.385155 14836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:38.385185 14836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:38.386684 14836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:38.386885 14836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:38.387053 14836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:38.391850 14844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:38.391934 14841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:38.392007 14836 server_base.cc:1034] running on GCE node
W20260929 19:09:38.391850 14842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:38.392522 14836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:38.392807 14836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:38.393947 14836 hybrid_clock.cc:648] HybridClock initialized: now 1790708978393933 us; error 30 us; skew 500 ppm
Sep 29 19:09:38 dist-test-slave-xz51 krb5kdc[14823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708978, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:38.402427 14836 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:38.402858 14836 webserver.cc:460] Webserver started at http://127.0.0.1:46221/ using document root <none> and password file <none>
I20260929 19:09:38.403039 14836 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:38.403079 14836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:38.403167 14836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:38.404165 14836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "b7e155bb158d4572a4d994f099610774"
format_stamp: "Formatted at 2026-09-29 19:09:38 on dist-test-slave-xz51"
server_key: "80e722ab29b4ce24cda038000016aa44"
server_key_iv: "86bff6864042b16d43ee51acfc1633c5"
server_key_version: "encryptionkey@0"
I20260929 19:09:38.404470 14836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "b7e155bb158d4572a4d994f099610774"
format_stamp: "Formatted at 2026-09-29 19:09:38 on dist-test-slave-xz51"
server_key: "80e722ab29b4ce24cda038000016aa44"
server_key_iv: "86bff6864042b16d43ee51acfc1633c5"
server_key_version: "encryptionkey@0"
I20260929 19:09:38.405717 14836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:38.406520 14851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:38.406697 14836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:38.406802 14836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "b7e155bb158d4572a4d994f099610774"
format_stamp: "Formatted at 2026-09-29 19:09:38 on dist-test-slave-xz51"
server_key: "80e722ab29b4ce24cda038000016aa44"
server_key_iv: "86bff6864042b16d43ee51acfc1633c5"
server_key_version: "encryptionkey@0"
I20260929 19:09:38.406877 14836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:38.420838 14836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:38.422749 14836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:38.422919 14836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:38.426879 14836 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38759
I20260929 19:09:38.426913 14903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38759 every 8 connection(s)
I20260929 19:09:38.427268 14836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:38.428189 14904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:38.430508 14904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774: Bootstrap starting.
I20260929 19:09:38.431084 14904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:38.431383 14904 log.cc:824] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:38.432132 14904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774: No bootstrap required, opened a new log
I20260929 19:09:38.433774 14904 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e155bb158d4572a4d994f099610774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260929 19:09:38.433945 14904 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:38.433970 14904 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e155bb158d4572a4d994f099610774, State: Initialized, Role: FOLLOWER
I20260929 19:09:38.434126 14904 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e155bb158d4572a4d994f099610774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260929 19:09:38.434191 14904 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:38.434213 14904 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:38.434283 14904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:38.434844 14904 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e155bb158d4572a4d994f099610774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260929 19:09:38.434958 14904 leader_election.cc:302] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7e155bb158d4572a4d994f099610774; no voters:
I20260929 19:09:38.435160 14904 leader_election.cc:288] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:38.435247 14907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:38.435416 14907 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [term 1 LEADER]: Becoming Leader. State: Replica: b7e155bb158d4572a4d994f099610774, State: Running, Role: LEADER
I20260929 19:09:38.435495 14904 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:38.435529 14907 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e155bb158d4572a4d994f099610774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260929 19:09:38.435981 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14836
I20260929 19:09:38.436097 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:38.436097 14909 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e155bb158d4572a4d994f099610774. Latest consensus state: current_term: 1 leader_uuid: "b7e155bb158d4572a4d994f099610774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e155bb158d4572a4d994f099610774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } } }
I20260929 19:09:38.436170 14909 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:38.436321 26939 external_mini_cluster.cc:1428] Setting key aacd0881039ee40ee78a122a2a3c806e
I20260929 19:09:38.436519 14908 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7e155bb158d4572a4d994f099610774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e155bb158d4572a4d994f099610774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } } }
I20260929 19:09:38.436604 14908 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774 [sys.catalog]: This master's current role is: LEADER
W20260929 19:09:38.437638 14925 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:09:38.437695 14925 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:09:38.437745 14916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:38.437937 14916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:38.442006 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:38.436833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42355 (local address 127.0.0.1:38759)
0929 19:09:38.436994 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:38.437004 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:38.437039 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:09:38.437151 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:38.437158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:38.437242 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:38.437416 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:38.437661 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:38.439062 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:38.439799 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:38.440442 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:38.440553 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:38.441252 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:38.441256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:38.441257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:38.441482 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:38.441523 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:38.441539 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:38.441624 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:38.441718 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 19:09:38.441749 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
I20260929 19:09:38.442283 14916 catalog_manager.cc:1347] Generated new cluster ID: 4eac2ca5883b4327a85d2822442e465c
I20260929 19:09:38.442333 14916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:38.442960 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:38.448036 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:38.444523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40929 (local address 127.0.0.1:38759)
0929 19:09:38.444558 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:38.444560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:38.444687 (+ 127us) server_negotiation.cc:407] Connection header received
0929 19:09:38.444745 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:38.444745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:38.444757 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:38.444802 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:38.444945 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:38.445677 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:38.446717 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:38.447045 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:38.447105 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:38.447788 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:38.447789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:38.447789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:38.447793 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:38.447806 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:38.447810 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:38.447865 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:38.447955 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 19:09:38.447966 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:09:38.468245 14916 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:38.469108 14916 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:38.477839 14916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e155bb158d4572a4d994f099610774: Generated new TSK 0
I20260929 19:09:38.478063 14916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:38.481225 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14836
W20260929 19:09:38.488632 14821 connection.cc:535] client connection to 127.0.0.1:38759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38759): Connection reset by peer (error 104)
2026-09-29T19:09:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:38 dist-test-slave-xz51 krb5kdc[14935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:38 dist-test-slave-xz51 krb5kdc[14935](info): set up 2 sockets
Sep 29 19:09:38 dist-test-slave-xz51 krb5kdc[14935](info): commencing operation
krb5kdc: starting...
W20260929 19:09:39.518065 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:39 dist-test-slave-xz51 krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:39.640606 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33481
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:39.722113 14948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:39.722329 14948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:39.722352 14948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:39.723485 14948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:39.723527 14948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:39.723541 14948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:39.723553 14948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:39.723567 14948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:39.725034 14948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33481
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:39.725234 14948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:39.725401 14948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:39.730250 14953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:39.730250 14954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:39.730260 14956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:39.730415 14948 server_base.cc:1034] running on GCE node
I20260929 19:09:39.730791 14948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:39.731091 14948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:39.732263 14948 hybrid_clock.cc:648] HybridClock initialized: now 1790708979732248 us; error 27 us; skew 500 ppm
Sep 29 19:09:39 dist-test-slave-xz51 krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:39.734807 14948 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:39.735246 14948 webserver.cc:460] Webserver started at http://127.0.0.1:39857/ using document root <none> and password file <none>
I20260929 19:09:39.735442 14948 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:39.735482 14948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:39.735569 14948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:39.736564 14948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "4c5985f9c68f4fa390611b96c51838d5"
format_stamp: "Formatted at 2026-09-29 19:09:39 on dist-test-slave-xz51"
server_key: "996d68e1185332473be07572d22cc1ff"
server_key_iv: "aa146a6d9171050662266d4cc671f860"
server_key_version: "encryptionkey@0"
I20260929 19:09:39.736852 14948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "4c5985f9c68f4fa390611b96c51838d5"
format_stamp: "Formatted at 2026-09-29 19:09:39 on dist-test-slave-xz51"
server_key: "996d68e1185332473be07572d22cc1ff"
server_key_iv: "aa146a6d9171050662266d4cc671f860"
server_key_version: "encryptionkey@0"
I20260929 19:09:39.738269 14948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:09:39.739096 14963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:39.739311 14948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:39.739413 14948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "4c5985f9c68f4fa390611b96c51838d5"
format_stamp: "Formatted at 2026-09-29 19:09:39 on dist-test-slave-xz51"
server_key: "996d68e1185332473be07572d22cc1ff"
server_key_iv: "aa146a6d9171050662266d4cc671f860"
server_key_version: "encryptionkey@0"
I20260929 19:09:39.739491 14948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:39.743124 14948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:39.745558 14948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:39.745699 14948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:39.749537 14948 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33481
I20260929 19:09:39.749569 15015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33481 every 8 connection(s)
I20260929 19:09:39.750015 14948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:39.751051 15016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:39.753314 15016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5: Bootstrap starting.
I20260929 19:09:39.754004 15016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:39.754293 15016 log.cc:824] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:39.754983 15016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5: No bootstrap required, opened a new log
I20260929 19:09:39.755482 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 14948
I20260929 19:09:39.755614 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:39.755801 26939 external_mini_cluster.cc:1428] Setting key b34742cb3279186d11ca5f58f806ebd5
I20260929 19:09:39.757462 15016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5985f9c68f4fa390611b96c51838d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260929 19:09:39.757675 15016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:39.757710 15016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5985f9c68f4fa390611b96c51838d5, State: Initialized, Role: FOLLOWER
I20260929 19:09:39.757876 15016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5985f9c68f4fa390611b96c51838d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260929 19:09:39.758005 15016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:39.758059 15016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:39.758113 15016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:39.758906 15016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5985f9c68f4fa390611b96c51838d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260929 19:09:39.758994 15016 leader_election.cc:302] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c5985f9c68f4fa390611b96c51838d5; no voters:
I20260929 19:09:39.759011 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:39.756444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37037 (local address 127.0.0.1:33481)
0929 19:09:39.756653 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:39.756664 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:39.756699 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:09:39.757673 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:39.757681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:39.757719 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:39.757910 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:39.758521 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:39.758524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:39.758524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:39.758528 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:39.758566 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:39.758577 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:39.758633 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:39.758726 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 19:09:39.758782 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260929 19:09:39.759177 15016 leader_election.cc:288] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:39.759239 15021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:39.759374 15021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [term 1 LEADER]: Becoming Leader. State: Replica: 4c5985f9c68f4fa390611b96c51838d5, State: Running, Role: LEADER
I20260929 19:09:39.759509 15021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5985f9c68f4fa390611b96c51838d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260929 19:09:39.759608 15016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:39.759980 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:39.759999 15022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c5985f9c68f4fa390611b96c51838d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5985f9c68f4fa390611b96c51838d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } } }
I20260929 19:09:39.760078 15022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:39.761382 15023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c5985f9c68f4fa390611b96c51838d5. Latest consensus state: current_term: 1 leader_uuid: "4c5985f9c68f4fa390611b96c51838d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5985f9c68f4fa390611b96c51838d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } } }
W20260929 19:09:39.761497 15039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260929 19:09:39.761772 15023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5 [sys.catalog]: This master's current role is: LEADER
W20260929 19:09:39.761790 15039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:09:39.761907 15026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:39.762100 15026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:39.763434 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:39.762204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60531 (local address 127.0.0.1:33481)
0929 19:09:39.762238 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:39.762239 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:39.762468 (+ 229us) server_negotiation.cc:407] Connection header received
0929 19:09:39.762510 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:39.762511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:39.762516 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:39.762586 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:39.763159 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:39.763160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:39.763160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:39.763164 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:39.763179 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:39.763184 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:39.763243 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:39.763340 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:09:39.763355 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 19:09:39.763626 15026 catalog_manager.cc:1347] Generated new cluster ID: 676d1dc0feed47a79b0dd4b9614f0546
I20260929 19:09:39.763679 15026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:39.788882 15026 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:39.789103 15026 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:39.795881 15026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c5985f9c68f4fa390611b96c51838d5: Generated new TSK 0
I20260929 19:09:39.796132 15026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:39.805485 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 14948
2026-09-29T19:09:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:39 dist-test-slave-xz51 krb5kdc[15047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:39 dist-test-slave-xz51 krb5kdc[15047](info): set up 2 sockets
Sep 29 19:09:39 dist-test-slave-xz51 krb5kdc[15047](info): commencing operation
krb5kdc: starting...
W20260929 19:09:40.842157 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:40 dist-test-slave-xz51 krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:40.958874 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:41.038821 15060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:41.039027 15060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:41.039052 15060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:41.040190 15060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:41.040233 15060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:41.040248 15060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:41.040261 15060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:41.040318 15060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:41.041759 15060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:41.042073 15060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:41.042351 15060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:41.047119 15066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:41.047188 15068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:41.047092 15065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:41.047295 15060 server_base.cc:1034] running on GCE node
I20260929 19:09:41.047518 15060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:41.047765 15060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:41.048962 15060 hybrid_clock.cc:648] HybridClock initialized: now 1790708981048926 us; error 45 us; skew 500 ppm
Sep 29 19:09:41 dist-test-slave-xz51 krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708981, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:41.051600 15060 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:41.052006 15060 webserver.cc:460] Webserver started at http://127.0.0.1:40815/ using document root <none> and password file <none>
I20260929 19:09:41.052199 15060 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:41.052243 15060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:41.052382 15060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:41.053442 15060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "e9dba523f2a94de5a97f8b2218425163"
format_stamp: "Formatted at 2026-09-29 19:09:41 on dist-test-slave-xz51"
server_key: "a170ee147b65b700393faef9a049f354"
server_key_iv: "aeac1549ef05b2bac3da3e56fb477c94"
server_key_version: "encryptionkey@0"
I20260929 19:09:41.053752 15060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "e9dba523f2a94de5a97f8b2218425163"
format_stamp: "Formatted at 2026-09-29 19:09:41 on dist-test-slave-xz51"
server_key: "a170ee147b65b700393faef9a049f354"
server_key_iv: "aeac1549ef05b2bac3da3e56fb477c94"
server_key_version: "encryptionkey@0"
I20260929 19:09:41.055099 15060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:41.055851 15075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:41.056022 15060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:09:41.056106 15060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "e9dba523f2a94de5a97f8b2218425163"
format_stamp: "Formatted at 2026-09-29 19:09:41 on dist-test-slave-xz51"
server_key: "a170ee147b65b700393faef9a049f354"
server_key_iv: "aeac1549ef05b2bac3da3e56fb477c94"
server_key_version: "encryptionkey@0"
I20260929 19:09:41.056156 15060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:41.071877 15060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:41.074262 15060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:41.074383 15060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:41.078363 15060 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46875
I20260929 19:09:41.078419 15127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46875 every 8 connection(s)
I20260929 19:09:41.078769 15060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:41.079672 15128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:41.081904 15128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163: Bootstrap starting.
I20260929 19:09:41.082496 15128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:41.082789 15128 log.cc:824] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:41.083501 15128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163: No bootstrap required, opened a new log
I20260929 19:09:41.083494 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15060
I20260929 19:09:41.083577 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:41.083770 26939 external_mini_cluster.cc:1428] Setting key 8b5ac43e514f9d2a131584d38a63d97e
I20260929 19:09:41.085338 15128 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dba523f2a94de5a97f8b2218425163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260929 19:09:41.085511 15128 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:41.085578 15128 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9dba523f2a94de5a97f8b2218425163, State: Initialized, Role: FOLLOWER
I20260929 19:09:41.085763 15128 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dba523f2a94de5a97f8b2218425163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260929 19:09:41.085892 15128 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:41.085938 15128 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:41.085994 15128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:41.086859 15128 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dba523f2a94de5a97f8b2218425163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260929 19:09:41.086982 15128 leader_election.cc:302] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9dba523f2a94de5a97f8b2218425163; no voters:
I20260929 19:09:41.087188 15128 leader_election.cc:288] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:41.087216 15133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:41.087324 15133 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [term 1 LEADER]: Becoming Leader. State: Replica: e9dba523f2a94de5a97f8b2218425163, State: Running, Role: LEADER
I20260929 19:09:41.087458 15128 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:41.087459 15133 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dba523f2a94de5a97f8b2218425163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260929 19:09:41.087932 15135 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9dba523f2a94de5a97f8b2218425163. Latest consensus state: current_term: 1 leader_uuid: "e9dba523f2a94de5a97f8b2218425163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dba523f2a94de5a97f8b2218425163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260929 19:09:41.087939 15134 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9dba523f2a94de5a97f8b2218425163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dba523f2a94de5a97f8b2218425163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260929 19:09:41.088129 15135 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:41.088137 15134 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:41.088373 15142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:41.088976 15142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:41.090731 15142 catalog_manager.cc:1347] Generated new cluster ID: db563d1112154b048cccb7b8363faff3
I20260929 19:09:41.090790 15142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:09:41 dist-test-slave-xz51 krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:09:41.095728 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:41.084275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48325 (local address 127.0.0.1:46875)
0929 19:09:41.084459 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:41.084469 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:41.084499 (+ 30us) server_negotiation.cc:407] Connection header received
0929 19:09:41.085324 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:41.085331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:41.085379 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:41.085518 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:41.086306 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:41.087349 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:41.088409 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:41.089032 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:41.089509 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:41.091951 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:41.091955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:41.091955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:41.091965 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:41.093916 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:41.094266 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:41.094268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:41.094269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:41.094384 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:41.094558 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:41.094558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:41.094558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:41.094824 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:41.095346 (+ 522us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:41.095412 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:09:41.095537 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260929 19:09:41.096441 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:41.104637 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:41.097750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35603 (local address 127.0.0.1:46875)
0929 19:09:41.097779 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:41.097781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:41.098008 (+ 227us) server_negotiation.cc:407] Connection header received
0929 19:09:41.098092 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:41.098093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:41.098112 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:41.098138 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:41.098728 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:41.099449 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:41.100382 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:41.100689 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:41.100723 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:41.102099 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:41.102100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:41.102100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:41.102107 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:41.103570 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:41.103913 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:41.103914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:41.103915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:41.103961 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:41.104124 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:41.104125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:41.104125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:41.104227 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:41.104302 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:41.104481 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 19:09:41.104550 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260929 19:09:41.108796 15142 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:41.109500 15142 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:41.117286 15142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9dba523f2a94de5a97f8b2218425163: Generated new TSK 0
I20260929 19:09:41.117504 15142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:41.127282 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15060
2026-09-29T19:09:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:41 dist-test-slave-xz51 krb5kdc[15159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:41 dist-test-slave-xz51 krb5kdc[15159](info): set up 2 sockets
Sep 29 19:09:41 dist-test-slave-xz51 krb5kdc[15159](info): commencing operation
krb5kdc: starting...
W20260929 19:09:42.162055 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:42 dist-test-slave-xz51 krb5kdc[15159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:42.280086 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42587
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:42.359447 15172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:42.359700 15172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:42.359740 15172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:42.360913 15172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:42.360989 15172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:42.361012 15172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:42.361032 15172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:42.361063 15172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:42.362587 15172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:42.362882 15172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:42.363116 15172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:42.367998 15178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:42.368011 15180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:42.368029 15172 server_base.cc:1034] running on GCE node
W20260929 19:09:42.368021 15177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:42.368567 15172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:42.368891 15172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:42.370060 15172 hybrid_clock.cc:648] HybridClock initialized: now 1790708982370038 us; error 35 us; skew 500 ppm
Sep 29 19:09:42 dist-test-slave-xz51 krb5kdc[15159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:42.372469 15172 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:42.372912 15172 webserver.cc:460] Webserver started at http://127.0.0.1:39787/ using document root <none> and password file <none>
I20260929 19:09:42.373158 15172 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:42.373227 15172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:42.373394 15172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:42.374496 15172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "0884d129705f44b79ede8b5705baacca"
format_stamp: "Formatted at 2026-09-29 19:09:42 on dist-test-slave-xz51"
server_key: "7f942f8b59125733873ef5233bc18d84"
server_key_iv: "6366ab9aefad3d792bc0f87cf2acb1e0"
server_key_version: "encryptionkey@0"
I20260929 19:09:42.374846 15172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "0884d129705f44b79ede8b5705baacca"
format_stamp: "Formatted at 2026-09-29 19:09:42 on dist-test-slave-xz51"
server_key: "7f942f8b59125733873ef5233bc18d84"
server_key_iv: "6366ab9aefad3d792bc0f87cf2acb1e0"
server_key_version: "encryptionkey@0"
I20260929 19:09:42.376241 15172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:42.377045 15187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:42.377215 15172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:42.377316 15172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "0884d129705f44b79ede8b5705baacca"
format_stamp: "Formatted at 2026-09-29 19:09:42 on dist-test-slave-xz51"
server_key: "7f942f8b59125733873ef5233bc18d84"
server_key_iv: "6366ab9aefad3d792bc0f87cf2acb1e0"
server_key_version: "encryptionkey@0"
I20260929 19:09:42.377398 15172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:42.403208 15172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:42.405197 15172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:42.405375 15172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:42.409658 15172 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42587
I20260929 19:09:42.409698 15239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42587 every 8 connection(s)
I20260929 19:09:42.410115 15172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:42.411084 15240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:42.413372 15240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca: Bootstrap starting.
I20260929 19:09:42.414000 15240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:42.414286 15240 log.cc:824] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:42.414651 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15172
I20260929 19:09:42.414791 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:42.414989 26939 external_mini_cluster.cc:1428] Setting key 55be05a173387d19ad14df0911eba7ae
I20260929 19:09:42.415086 15240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca: No bootstrap required, opened a new log
I20260929 19:09:42.417011 15240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0884d129705f44b79ede8b5705baacca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260929 19:09:42.417145 15240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:42.417167 15240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0884d129705f44b79ede8b5705baacca, State: Initialized, Role: FOLLOWER
I20260929 19:09:42.417309 15240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0884d129705f44b79ede8b5705baacca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260929 19:09:42.417379 15240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:42.417438 15240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:42.417510 15240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:42.418148 15240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0884d129705f44b79ede8b5705baacca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260929 19:09:42.418236 15240 leader_election.cc:302] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0884d129705f44b79ede8b5705baacca; no voters:
I20260929 19:09:42.418434 15240 leader_election.cc:288] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:42.418460 15245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:42.418653 15245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [term 1 LEADER]: Becoming Leader. State: Replica: 0884d129705f44b79ede8b5705baacca, State: Running, Role: LEADER
I20260929 19:09:42.418686 15240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:42.418766 15245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0884d129705f44b79ede8b5705baacca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260929 19:09:42.419111 15245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0884d129705f44b79ede8b5705baacca. Latest consensus state: current_term: 1 leader_uuid: "0884d129705f44b79ede8b5705baacca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0884d129705f44b79ede8b5705baacca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } } }
I20260929 19:09:42.419121 15246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0884d129705f44b79ede8b5705baacca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0884d129705f44b79ede8b5705baacca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } } }
I20260929 19:09:42.419209 15245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:42.419211 15246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:42.419724 15249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:42.420203 15249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:42.421964 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:42.415521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38923 (local address 127.0.0.1:42587)
0929 19:09:42.415694 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:42.415704 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:42.415736 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:09:42.416326 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:42.416333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:42.416399 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:42.416554 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:42.416733 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:42.418185 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:42.419458 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:42.420066 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:42.420177 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:42.421043 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:42.421048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:42.421048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:42.421263 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:42.421303 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:42.421317 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:42.421394 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:42.421631 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 19:09:42.421714 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260929 19:09:42.422065 15249 catalog_manager.cc:1347] Generated new cluster ID: eddac32ad9dc4d5191d0e3922c770786
I20260929 19:09:42.422106 15249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:42.422979 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:42.429451 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:42.425911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49223 (local address 127.0.0.1:42587)
0929 19:09:42.425955 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:42.425957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:42.425966 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:09:42.425987 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:42.425988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:42.425999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:42.426095 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:42.426303 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:42.427082 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:42.428097 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:42.428431 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:42.428508 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:42.429191 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:42.429192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:42.429193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:42.429198 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:42.429214 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:42.429219 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:42.429280 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:42.429335 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 19:09:42.429366 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:09:42.453421 15249 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:42.454080 15249 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:42.462466 15249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0884d129705f44b79ede8b5705baacca: Generated new TSK 0
I20260929 19:09:42.462685 15249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:42.465685 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15172
2026-09-29T19:09:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:42 dist-test-slave-xz51 krb5kdc[15271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:42 dist-test-slave-xz51 krb5kdc[15271](info): set up 2 sockets
Sep 29 19:09:42 dist-test-slave-xz51 krb5kdc[15271](info): commencing operation
krb5kdc: starting...
W20260929 19:09:43.530049 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:43 dist-test-slave-xz51 krb5kdc[15271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:43.648584 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:43.729118 15284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:43.729329 15284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:43.729352 15284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:43.730474 15284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:43.730510 15284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:43.730525 15284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:43.730537 15284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:43.730551 15284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:43.731961 15284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:43.732149 15284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:43.732327 15284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:43.737037 15289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:43.737082 15290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:43.737048 15292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:43.737442 15284 server_base.cc:1034] running on GCE node
I20260929 19:09:43.737680 15284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:43.738008 15284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:43.739174 15284 hybrid_clock.cc:648] HybridClock initialized: now 1790708983739162 us; error 27 us; skew 500 ppm
Sep 29 19:09:43 dist-test-slave-xz51 krb5kdc[15271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708983, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:43.749244 15284 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:43.749774 15284 webserver.cc:460] Webserver started at http://127.0.0.1:36689/ using document root <none> and password file <none>
I20260929 19:09:43.750072 15284 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:43.750146 15284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:43.750308 15284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:43.751399 15284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "2a3df463434d4aee9c468fc3fee3ecd0"
format_stamp: "Formatted at 2026-09-29 19:09:43 on dist-test-slave-xz51"
server_key: "8c0815e14f61965fa04ced0c98500574"
server_key_iv: "874b7ae7b65f9d0147dab9d56e05e706"
server_key_version: "encryptionkey@0"
I20260929 19:09:43.751780 15284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "2a3df463434d4aee9c468fc3fee3ecd0"
format_stamp: "Formatted at 2026-09-29 19:09:43 on dist-test-slave-xz51"
server_key: "8c0815e14f61965fa04ced0c98500574"
server_key_iv: "874b7ae7b65f9d0147dab9d56e05e706"
server_key_version: "encryptionkey@0"
I20260929 19:09:43.753163 15284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:43.754048 15299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:43.754225 15284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:43.754321 15284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "2a3df463434d4aee9c468fc3fee3ecd0"
format_stamp: "Formatted at 2026-09-29 19:09:43 on dist-test-slave-xz51"
server_key: "8c0815e14f61965fa04ced0c98500574"
server_key_iv: "874b7ae7b65f9d0147dab9d56e05e706"
server_key_version: "encryptionkey@0"
I20260929 19:09:43.754408 15284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:43.757719 15284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:43.759573 15284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:43.759738 15284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:43.763720 15284 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43813
I20260929 19:09:43.763754 15351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43813 every 8 connection(s)
I20260929 19:09:43.764081 15284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:43.765046 15352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:43.767401 15352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0: Bootstrap starting.
I20260929 19:09:43.767913 15352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:43.768142 15352 log.cc:824] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:43.768764 15352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0: No bootstrap required, opened a new log
I20260929 19:09:43.770382 15352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260929 19:09:43.770520 15352 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:43.770542 15352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a3df463434d4aee9c468fc3fee3ecd0, State: Initialized, Role: FOLLOWER
I20260929 19:09:43.770677 15352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260929 19:09:43.770758 15352 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:43.770781 15352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:43.770809 15352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:43.771275 15352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260929 19:09:43.771368 15352 leader_election.cc:302] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a3df463434d4aee9c468fc3fee3ecd0; no voters:
I20260929 19:09:43.771504 15352 leader_election.cc:288] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:43.771569 15355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:43.771744 15355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [term 1 LEADER]: Becoming Leader. State: Replica: 2a3df463434d4aee9c468fc3fee3ecd0, State: Running, Role: LEADER
I20260929 19:09:43.771778 15352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:43.771832 15355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260929 19:09:43.772224 15356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } } }
I20260929 19:09:43.772290 15357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a3df463434d4aee9c468fc3fee3ecd0. Latest consensus state: current_term: 1 leader_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3df463434d4aee9c468fc3fee3ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } } }
I20260929 19:09:43.772379 15356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:43.772398 15357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:43.772774 15360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:43.773291 15360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:43.773290 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15284
I20260929 19:09:43.773398 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:43.773610 26939 external_mini_cluster.cc:1428] Setting key a6223fcb654bbc758a66c726b27a2f5e
I20260929 19:09:43.774991 15360 catalog_manager.cc:1347] Generated new cluster ID: 4849f5b212314c5aae4308eeaa90f7f9
I20260929 19:09:43.775039 15360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:43.775981 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:43.774082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52535 (local address 127.0.0.1:43813)
0929 19:09:43.774247 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:43.774256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:43.774437 (+ 181us) server_negotiation.cc:407] Connection header received
0929 19:09:43.774635 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:43.774643 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:43.774678 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:43.774822 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:43.775528 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:43.775532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:43.775532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:43.775536 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:43.775583 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:43.775594 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:43.775661 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:43.775733 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 19:09:43.775776 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260929 19:09:43.778699 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:43.787957 15360 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:43.788151 15360 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:43.788859 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:43.780192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41217 (local address 127.0.0.1:43813)
0929 19:09:43.780244 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:43.780245 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:43.787753 (+ 7508us) server_negotiation.cc:407] Connection header received
0929 19:09:43.787857 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:43.787858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:43.787864 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:43.787916 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:43.788631 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:43.788632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:43.788633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:43.788637 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:43.788653 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:43.788658 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:43.788718 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:43.788770 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 19:09:43.788783 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:09:43.796280 15360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a3df463434d4aee9c468fc3fee3ecd0: Generated new TSK 0
I20260929 19:09:43.796517 15360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:43.803602 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15284
2026-09-29T19:09:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:43 dist-test-slave-xz51 krb5kdc[15383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:43 dist-test-slave-xz51 krb5kdc[15383](info): set up 2 sockets
Sep 29 19:09:43 dist-test-slave-xz51 krb5kdc[15383](info): commencing operation
krb5kdc: starting...
W20260929 19:09:44.829979 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:44 dist-test-slave-xz51 krb5kdc[15383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:44.950703 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:45.030922 15396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:45.031134 15396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:45.031157 15396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:45.032256 15396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:45.032294 15396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:45.032308 15396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:45.032321 15396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:45.032333 15396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:45.033715 15396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:45.033980 15396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:45.034166 15396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:09:45.039017 15396 server_base.cc:1034] running on GCE node
W20260929 19:09:45.038990 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:45.038966 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:45.038981 15401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:45.039536 15396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:45.039819 15396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:45.040964 15396 hybrid_clock.cc:648] HybridClock initialized: now 1790708985040951 us; error 29 us; skew 500 ppm
Sep 29 19:09:45 dist-test-slave-xz51 krb5kdc[15383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:45.043511 15396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:45.043939 15396 webserver.cc:460] Webserver started at http://127.0.0.1:43433/ using document root <none> and password file <none>
I20260929 19:09:45.044159 15396 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:45.044204 15396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:45.044368 15396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:45.045436 15396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac"
format_stamp: "Formatted at 2026-09-29 19:09:45 on dist-test-slave-xz51"
server_key: "a10d95c3c08981b0aa08c04de7bea839"
server_key_iv: "5b494b70116f2ada220f99bb759938ce"
server_key_version: "encryptionkey@0"
I20260929 19:09:45.045785 15396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac"
format_stamp: "Formatted at 2026-09-29 19:09:45 on dist-test-slave-xz51"
server_key: "a10d95c3c08981b0aa08c04de7bea839"
server_key_iv: "5b494b70116f2ada220f99bb759938ce"
server_key_version: "encryptionkey@0"
I20260929 19:09:45.047099 15396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:45.048079 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:45.048225 15396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:45.048312 15396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac"
format_stamp: "Formatted at 2026-09-29 19:09:45 on dist-test-slave-xz51"
server_key: "a10d95c3c08981b0aa08c04de7bea839"
server_key_iv: "5b494b70116f2ada220f99bb759938ce"
server_key_version: "encryptionkey@0"
I20260929 19:09:45.048379 15396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:45.079308 15396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:45.081727 15396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:45.081876 15396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:45.085808 15396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42763
I20260929 19:09:45.085866 15463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42763 every 8 connection(s)
I20260929 19:09:45.086257 15396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:45.087095 15464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:45.089337 15464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac: Bootstrap starting.
I20260929 19:09:45.089957 15464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:45.090237 15464 log.cc:824] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:45.090934 15464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac: No bootstrap required, opened a new log
I20260929 19:09:45.092578 15464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260929 19:09:45.092748 15464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:45.092794 15464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c0d77f421c45e3813ed9b6f4cfa6ac, State: Initialized, Role: FOLLOWER
I20260929 19:09:45.092963 15464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260929 19:09:45.093048 15464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:45.093092 15464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:45.093144 15464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:45.093675 15464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260929 19:09:45.093793 15464 leader_election.cc:302] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27c0d77f421c45e3813ed9b6f4cfa6ac; no voters:
I20260929 19:09:45.094004 15464 leader_election.cc:288] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:45.094036 15467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:45.094208 15467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [term 1 LEADER]: Becoming Leader. State: Replica: 27c0d77f421c45e3813ed9b6f4cfa6ac, State: Running, Role: LEADER
I20260929 19:09:45.094255 15464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:45.094300 15467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260929 19:09:45.094667 15469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27c0d77f421c45e3813ed9b6f4cfa6ac. Latest consensus state: current_term: 1 leader_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } } }
I20260929 19:09:45.094759 15469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:45.094805 15468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c0d77f421c45e3813ed9b6f4cfa6ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } } }
I20260929 19:09:45.094898 15468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:45.094996 15475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:45.095558 15475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:45.095842 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15396
I20260929 19:09:45.095958 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:45.096148 26939 external_mini_cluster.cc:1428] Setting key 8b27bfe9eaa3ab9a8022ea67cd948213
I20260929 19:09:45.097262 15475 catalog_manager.cc:1347] Generated new cluster ID: 0328a4f014de4bd0a6deeca663aef68f
I20260929 19:09:45.097316 15475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:09:45 dist-test-slave-xz51 krb5kdc[15383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:09:45.108650 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:45.096639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:46139 (local address 127.0.0.1:42763)
0929 19:09:45.096994 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:45.097005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:45.097037 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:09:45.097149 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:45.097157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:45.097896 (+ 739us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:45.098037 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:45.098811 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:45.100217 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:45.101321 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:45.101949 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:45.102043 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:45.104886 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:45.104891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:45.104891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:45.104906 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:45.106919 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:45.107357 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:45.107360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:45.107360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:45.107475 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:45.107707 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:45.107708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:45.107709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:45.108026 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:45.108175 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:45.108300 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 19:09:45.108405 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":65,"threads_started":1}
I20260929 19:09:45.109987 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:45.116137 15475 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:45.116624 15475 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:45.118912 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:45.111470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:49693 (local address 127.0.0.1:42763)
0929 19:09:45.111510 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:45.111512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:45.111703 (+ 191us) server_negotiation.cc:407] Connection header received
0929 19:09:45.111736 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:45.111737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:45.111748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:45.111816 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:45.112389 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:45.113171 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:45.114165 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:45.114493 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:45.114565 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:45.115975 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:45.115976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:45.115977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:45.115984 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:45.117699 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:45.118130 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:45.118131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:45.118132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:45.118190 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:45.118409 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:45.118410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:45.118411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:45.118536 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:45.118651 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:45.118755 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 19:09:45.118816 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:09:45.125882 15475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27c0d77f421c45e3813ed9b6f4cfa6ac: Generated new TSK 0
I20260929 19:09:45.126092 15475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:45.132365 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15396
W20260929 19:09:45.139066 15381 connection.cc:535] client connection to 127.0.0.1:42763 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42763): Connection reset by peer (error 104)
2026-09-29T19:09:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:45 dist-test-slave-xz51 krb5kdc[15495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:45 dist-test-slave-xz51 krb5kdc[15495](info): set up 2 sockets
Sep 29 19:09:45 dist-test-slave-xz51 krb5kdc[15495](info): commencing operation
krb5kdc: starting...
W20260929 19:09:46.166097 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:46 dist-test-slave-xz51 krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:46.289026 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33327
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:46.370975 15508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:46.371213 15508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:46.371254 15508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:46.372459 15508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:46.372539 15508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:46.372565 15508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:46.372596 15508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:46.372643 15508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:46.374213 15508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:46.374495 15508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:46.374727 15508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:46.379719 15514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:46.379843 15508 server_base.cc:1034] running on GCE node
W20260929 19:09:46.379730 15513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:46.379699 15516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:46.380159 15508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:46.380522 15508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:46.381670 15508 hybrid_clock.cc:648] HybridClock initialized: now 1790708986381649 us; error 38 us; skew 500 ppm
Sep 29 19:09:46 dist-test-slave-xz51 krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:46.384198 15508 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:46.384579 15508 webserver.cc:460] Webserver started at http://127.0.0.1:34379/ using document root <none> and password file <none>
I20260929 19:09:46.384775 15508 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:46.384814 15508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:46.384904 15508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:46.386010 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "2aa775b640084b07914c52bd5ca7c9a1"
format_stamp: "Formatted at 2026-09-29 19:09:46 on dist-test-slave-xz51"
server_key: "77b6ad6d452c6a897013469d6cd9e120"
server_key_iv: "4a73fcc3936d9e5045c7a0fb3363546a"
server_key_version: "encryptionkey@0"
I20260929 19:09:46.386318 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "2aa775b640084b07914c52bd5ca7c9a1"
format_stamp: "Formatted at 2026-09-29 19:09:46 on dist-test-slave-xz51"
server_key: "77b6ad6d452c6a897013469d6cd9e120"
server_key_iv: "4a73fcc3936d9e5045c7a0fb3363546a"
server_key_version: "encryptionkey@0"
I20260929 19:09:46.387552 15508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:46.388370 15523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:46.388574 15508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:46.388648 15508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "2aa775b640084b07914c52bd5ca7c9a1"
format_stamp: "Formatted at 2026-09-29 19:09:46 on dist-test-slave-xz51"
server_key: "77b6ad6d452c6a897013469d6cd9e120"
server_key_iv: "4a73fcc3936d9e5045c7a0fb3363546a"
server_key_version: "encryptionkey@0"
I20260929 19:09:46.388739 15508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:46.413100 15508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:46.415198 15508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:46.415346 15508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:46.419363 15508 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33327
I20260929 19:09:46.419438 15575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33327 every 8 connection(s)
I20260929 19:09:46.419768 15508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:46.420670 15576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:46.423035 15576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1: Bootstrap starting.
I20260929 19:09:46.423631 15576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:46.423942 15576 log.cc:824] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:46.424012 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15508
I20260929 19:09:46.424154 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:46.424335 26939 external_mini_cluster.cc:1428] Setting key 5d9c87476f0640a35a396cb746f3cb0a
I20260929 19:09:46.424782 15576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1: No bootstrap required, opened a new log
I20260929 19:09:46.427639 15576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa775b640084b07914c52bd5ca7c9a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33327 } }
I20260929 19:09:46.427810 15576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:46.427840 15576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aa775b640084b07914c52bd5ca7c9a1, State: Initialized, Role: FOLLOWER
I20260929 19:09:46.427964 15576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa775b640084b07914c52bd5ca7c9a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33327 } }
I20260929 19:09:46.428053 15576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:46.428089 15576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:46.428138 15576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:46.428694 15576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa775b640084b07914c52bd5ca7c9a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33327 } }
I20260929 19:09:46.428782 15576 leader_election.cc:302] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aa775b640084b07914c52bd5ca7c9a1; no voters:
I20260929 19:09:46.428972 15576 leader_election.cc:288] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:46.429020 15581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:46.429185 15581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 2aa775b640084b07914c52bd5ca7c9a1, State: Running, Role: LEADER
I20260929 19:09:46.429220 15576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:46.429324 15581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa775b640084b07914c52bd5ca7c9a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33327 } }
I20260929 19:09:46.429929 15582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aa775b640084b07914c52bd5ca7c9a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa775b640084b07914c52bd5ca7c9a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33327 } } }
I20260929 19:09:46.430188 15582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:46.430116 15583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aa775b640084b07914c52bd5ca7c9a1. Latest consensus state: current_term: 1 leader_uuid: "2aa775b640084b07914c52bd5ca7c9a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa775b640084b07914c52bd5ca7c9a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33327 } } }
I20260929 19:09:46.430388 15583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:46.430464 15591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:46.431262 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:46.424858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:57371 (local address 127.0.0.1:33327)
0929 19:09:46.425081 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:46.425091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:46.425124 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:09:46.426138 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:46.426148 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:46.426216 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:46.426375 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:46.426535 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:46.427966 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:46.428841 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:46.429467 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:46.429612 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:46.430415 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:46.430419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:46.430420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:46.430635 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:46.430681 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:46.430698 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:46.430788 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:46.430931 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 19:09:46.430960 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":82,"threads_started":1}
I20260929 19:09:46.431011 15591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:46.432340 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:46.434829 15591 catalog_manager.cc:1347] Generated new cluster ID: de0597482be6440db18d825611ef279d
I20260929 19:09:46.434888 15591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:46.437166 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:46.433640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:50321 (local address 127.0.0.1:33327)
0929 19:09:46.433703 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:46.433705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:46.433767 (+ 62us) server_negotiation.cc:407] Connection header received
0929 19:09:46.433867 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:46.433868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:46.433878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:46.433943 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:46.434070 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:46.434679 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:46.435714 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:46.436024 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:46.436086 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:46.436892 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:46.436894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:46.436894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:46.436898 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:46.436912 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:46.436915 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:46.436960 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:46.437089 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 19:09:46.437097 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:09:46.448611 15591 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:46.449355 15591 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:46.455720 15591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aa775b640084b07914c52bd5ca7c9a1: Generated new TSK 0
I20260929 19:09:46.455947 15591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:46.463680 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15508
W20260929 19:09:46.467698 15493 connection.cc:535] client connection to 127.0.0.1:33327 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33327): Connection reset by peer (error 104)
2026-09-29T19:09:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:46 dist-test-slave-xz51 krb5kdc[15607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:46 dist-test-slave-xz51 krb5kdc[15607](info): set up 2 sockets
Sep 29 19:09:46 dist-test-slave-xz51 krb5kdc[15607](info): commencing operation
krb5kdc: starting...
W20260929 19:09:47.518143 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:47 dist-test-slave-xz51 krb5kdc[15607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:47.638624 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:47.720932 15620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:47.721148 15620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:47.721172 15620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:47.722293 15620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:47.722332 15620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:47.722347 15620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:47.722359 15620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:47.722373 15620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:47.723902 15620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:47.724110 15620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:47.724282 15620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:47.729178 15626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:47.729156 15625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:47.729231 15628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:47.729841 15620 server_base.cc:1034] running on GCE node
I20260929 19:09:47.730123 15620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:47.730463 15620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:47.731608 15620 hybrid_clock.cc:648] HybridClock initialized: now 1790708987731588 us; error 37 us; skew 500 ppm
Sep 29 19:09:47 dist-test-slave-xz51 krb5kdc[15607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:47.734259 15620 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:47.734735 15620 webserver.cc:460] Webserver started at http://127.0.0.1:46565/ using document root <none> and password file <none>
I20260929 19:09:47.734977 15620 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:47.735023 15620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:47.735170 15620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:47.736232 15620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "fddbd570b54741289b7fa0e9380cedd0"
format_stamp: "Formatted at 2026-09-29 19:09:47 on dist-test-slave-xz51"
server_key: "7d3128775170fa1e2e691bd211a8be57"
server_key_iv: "409aa1f887485cd8e3ea5e791e212d03"
server_key_version: "encryptionkey@0"
I20260929 19:09:47.736591 15620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "fddbd570b54741289b7fa0e9380cedd0"
format_stamp: "Formatted at 2026-09-29 19:09:47 on dist-test-slave-xz51"
server_key: "7d3128775170fa1e2e691bd211a8be57"
server_key_iv: "409aa1f887485cd8e3ea5e791e212d03"
server_key_version: "encryptionkey@0"
I20260929 19:09:47.738166 15620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:09:47.738910 15635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:47.739130 15620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:47.739203 15620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "fddbd570b54741289b7fa0e9380cedd0"
format_stamp: "Formatted at 2026-09-29 19:09:47 on dist-test-slave-xz51"
server_key: "7d3128775170fa1e2e691bd211a8be57"
server_key_iv: "409aa1f887485cd8e3ea5e791e212d03"
server_key_version: "encryptionkey@0"
I20260929 19:09:47.739290 15620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:47.742738 15620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:47.745074 15620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:47.745234 15620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:47.749226 15620 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33915
I20260929 19:09:47.749300 15687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33915 every 8 connection(s)
I20260929 19:09:47.749629 15620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:47.750686 15688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:47.752981 15688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0: Bootstrap starting.
I20260929 19:09:47.753603 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15620
I20260929 19:09:47.753670 15688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:47.753741 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:47.754007 26939 external_mini_cluster.cc:1428] Setting key 571b025d7b5ad034044331f83b82947d
I20260929 19:09:47.754037 15688 log.cc:824] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:47.755026 15688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0: No bootstrap required, opened a new log
I20260929 19:09:47.756657 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:47.754607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:57107 (local address 127.0.0.1:33915)
0929 19:09:47.754785 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:47.754793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:47.754819 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:09:47.755320 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:47.755327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:47.755355 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:47.755501 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:47.756180 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:47.756183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:47.756183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:47.756186 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:47.756228 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:47.756238 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:47.756306 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:47.756399 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 19:09:47.756464 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260929 19:09:47.757704 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:47.757802 15688 raft_consensus.cc:348] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fddbd570b54741289b7fa0e9380cedd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260929 19:09:47.757998 15688 raft_consensus.cc:374] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:47.758031 15688 raft_consensus.cc:729] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fddbd570b54741289b7fa0e9380cedd0, State: Initialized, Role: FOLLOWER
I20260929 19:09:47.758165 15688 consensus_queue.cc:260] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fddbd570b54741289b7fa0e9380cedd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260929 19:09:47.758265 15688 raft_consensus.cc:388] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:47.758297 15688 raft_consensus.cc:482] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:47.758349 15688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:47.759162 15688 raft_consensus.cc:504] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fddbd570b54741289b7fa0e9380cedd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260929 19:09:47.759334 15688 leader_election.cc:302] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fddbd570b54741289b7fa0e9380cedd0; no voters:
I20260929 19:09:47.759531 15688 leader_election.cc:288] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:47.759570 15698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:47.759809 15698 raft_consensus.cc:686] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [term 1 LEADER]: Becoming Leader. State: Replica: fddbd570b54741289b7fa0e9380cedd0, State: Running, Role: LEADER
I20260929 19:09:47.759778 15688 sys_catalog.cc:564] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:47.759958 15698 consensus_queue.cc:237] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fddbd570b54741289b7fa0e9380cedd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260929 19:09:47.760452 15700 sys_catalog.cc:455] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fddbd570b54741289b7fa0e9380cedd0. Latest consensus state: current_term: 1 leader_uuid: "fddbd570b54741289b7fa0e9380cedd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fddbd570b54741289b7fa0e9380cedd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } } }
I20260929 19:09:47.760584 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:47.759296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:42315 (local address 127.0.0.1:33915)
0929 19:09:47.759386 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:47.759388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:47.759448 (+ 60us) server_negotiation.cc:407] Connection header received
0929 19:09:47.759530 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:47.759531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:47.759537 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:47.759598 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:47.760308 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:47.760310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:47.760310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:47.760316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:47.760331 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:47.760335 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:47.760391 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:47.760486 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 19:09:47.760501 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 19:09:47.760609 15700 sys_catalog.cc:458] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:47.761361 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fddbd570b54741289b7fa0e9380cedd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fddbd570b54741289b7fa0e9380cedd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } } }
I20260929 19:09:47.761438 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0 [sys.catalog]: This master's current role is: LEADER
W20260929 19:09:47.761715 15713 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:09:47.761754 15713 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:09:47.761842 15707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:47.762030 15707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:47.763440 15707 catalog_manager.cc:1347] Generated new cluster ID: 7280f090626140f68b366649458fb17a
I20260929 19:09:47.763490 15707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:47.771446 15707 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:47.771621 15707 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:47.774693 15707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fddbd570b54741289b7fa0e9380cedd0: Generated new TSK 0
I20260929 19:09:47.774868 15707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:47.782770 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15620
2026-09-29T19:09:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:47 dist-test-slave-xz51 krb5kdc[15719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:47 dist-test-slave-xz51 krb5kdc[15719](info): set up 2 sockets
Sep 29 19:09:47 dist-test-slave-xz51 krb5kdc[15719](info): commencing operation
krb5kdc: starting...
W20260929 19:09:48.874135 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:48 dist-test-slave-xz51 krb5kdc[15719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:48.992719 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38689
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:49.075579 15732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:49.075831 15732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:49.075878 15732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:49.077065 15732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:49.077136 15732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:49.077159 15732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:49.077191 15732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:49.077217 15732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:49.078784 15732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:49.079084 15732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:49.079329 15732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:49.084519 15738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:49.084532 15740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:49.084615 15737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:49.084744 15732 server_base.cc:1034] running on GCE node
I20260929 19:09:49.085145 15732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:49.085464 15732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:49.086634 15732 hybrid_clock.cc:648] HybridClock initialized: now 1790708989086616 us; error 51 us; skew 500 ppm
Sep 29 19:09:49 dist-test-slave-xz51 krb5kdc[15719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:49.096402 15732 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:49.096896 15732 webserver.cc:460] Webserver started at http://127.0.0.1:41729/ using document root <none> and password file <none>
I20260929 19:09:49.097221 15732 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:49.097286 15732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:49.097422 15732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:49.098608 15732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "21cdccb5d92d458493f3dd688bd7dda1"
format_stamp: "Formatted at 2026-09-29 19:09:49 on dist-test-slave-xz51"
server_key: "ba01cb860baa503f62b2854160d41af5"
server_key_iv: "d177c7799ff8b31eaaff6bbd18948a8d"
server_key_version: "encryptionkey@0"
I20260929 19:09:49.098985 15732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "21cdccb5d92d458493f3dd688bd7dda1"
format_stamp: "Formatted at 2026-09-29 19:09:49 on dist-test-slave-xz51"
server_key: "ba01cb860baa503f62b2854160d41af5"
server_key_iv: "d177c7799ff8b31eaaff6bbd18948a8d"
server_key_version: "encryptionkey@0"
I20260929 19:09:49.100544 15732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:49.101473 15747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:49.101703 15732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:49.101810 15732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "21cdccb5d92d458493f3dd688bd7dda1"
format_stamp: "Formatted at 2026-09-29 19:09:49 on dist-test-slave-xz51"
server_key: "ba01cb860baa503f62b2854160d41af5"
server_key_iv: "d177c7799ff8b31eaaff6bbd18948a8d"
server_key_version: "encryptionkey@0"
I20260929 19:09:49.101943 15732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:49.116346 15732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:49.118492 15732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:49.118643 15732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:49.124210 15732 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38689
I20260929 19:09:49.124269 15799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38689 every 8 connection(s)
I20260929 19:09:49.124707 15732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:49.125711 15800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:49.127754 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15732
I20260929 19:09:49.127889 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:49.128078 26939 external_mini_cluster.cc:1428] Setting key 902be1ac21807a154898af6b4afe30df
I20260929 19:09:49.129272 15800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1: Bootstrap starting.
I20260929 19:09:49.130005 15800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:49.130332 15800 log.cc:824] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:49.131096 15800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1: No bootstrap required, opened a new log
I20260929 19:09:49.133314 15800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdccb5d92d458493f3dd688bd7dda1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38689 } }
I20260929 19:09:49.133491 15800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:49.133533 15800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21cdccb5d92d458493f3dd688bd7dda1, State: Initialized, Role: FOLLOWER
I20260929 19:09:49.133670 15800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdccb5d92d458493f3dd688bd7dda1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38689 } }
I20260929 19:09:49.133764 15800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:49.133795 15800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:49.133860 15800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:49.134443 15800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdccb5d92d458493f3dd688bd7dda1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38689 } }
I20260929 19:09:49.134531 15800 leader_election.cc:302] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21cdccb5d92d458493f3dd688bd7dda1; no voters:
I20260929 19:09:49.134732 15800 leader_election.cc:288] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:49.134771 15805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:49.134884 15805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [term 1 LEADER]: Becoming Leader. State: Replica: 21cdccb5d92d458493f3dd688bd7dda1, State: Running, Role: LEADER
I20260929 19:09:49.135013 15800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:49.135021 15805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdccb5d92d458493f3dd688bd7dda1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38689 } }
I20260929 19:09:49.135377 15807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21cdccb5d92d458493f3dd688bd7dda1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdccb5d92d458493f3dd688bd7dda1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38689 } } }
I20260929 19:09:49.135365 15805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21cdccb5d92d458493f3dd688bd7dda1. Latest consensus state: current_term: 1 leader_uuid: "21cdccb5d92d458493f3dd688bd7dda1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdccb5d92d458493f3dd688bd7dda1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38689 } } }
I20260929 19:09:49.135555 15807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:49.135568 15805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1 [sys.catalog]: This master's current role is: LEADER
Sep 29 19:09:49 dist-test-slave-xz51 krb5kdc[15719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:09:49.135922 15814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:49.136488 15814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:49.140123 15803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:49.128625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:53675 (local address 127.0.0.1:38689)
0929 19:09:49.128841 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:49.128852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:49.128888 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:09:49.129441 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:49.129450 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:49.129525 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:49.129694 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:49.130441 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:49.131843 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:49.132595 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:49.133300 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:49.133427 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:49.136330 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:49.136334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:49.136335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:49.136348 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:49.138555 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:49.138956 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:49.138959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:49.138960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:49.139056 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:49.139293 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:49.139294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:49.139295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:49.139535 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:49.139706 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:49.139850 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 19:09:49.139953 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260929 19:09:49.141180 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:49.141913 15814 catalog_manager.cc:1347] Generated new cluster ID: 4e22b381fae743c4b27cb6693744286d
I20260929 19:09:49.141989 15814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:49.148698 15803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:49.142636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:51811 (local address 127.0.0.1:38689)
0929 19:09:49.142677 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:49.142679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:49.142739 (+ 60us) server_negotiation.cc:407] Connection header received
0929 19:09:49.142811 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:49.142812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:49.142822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:49.142887 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:49.143428 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:49.143934 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:49.144637 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:49.144855 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:49.144939 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:49.146186 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:49.146188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:49.146188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:49.146195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:49.147458 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:49.147858 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:49.147859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:49.147859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:49.147931 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:49.148155 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:49.148156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:49.148157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:49.148268 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:49.148377 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:49.148536 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 19:09:49.148598 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:09:49.157024 15814 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:49.157759 15814 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:49.162920 15814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21cdccb5d92d458493f3dd688bd7dda1: Generated new TSK 0
I20260929 19:09:49.163113 15814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:49.166821 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15732
W20260929 19:09:49.170882 15717 connection.cc:535] client connection to 127.0.0.1:38689 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38689): Connection reset by peer (error 104)
2026-09-29T19:09:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:49 dist-test-slave-xz51 krb5kdc[15831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:49 dist-test-slave-xz51 krb5kdc[15831](info): set up 2 sockets
Sep 29 19:09:49 dist-test-slave-xz51 krb5kdc[15831](info): commencing operation
krb5kdc: starting...
W20260929 19:09:50.210146 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:50 dist-test-slave-xz51 krb5kdc[15831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:50.329484 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:50.410672 15844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:50.410882 15844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:50.410905 15844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:50.412003 15844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:50.412040 15844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:50.412055 15844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:50.412067 15844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:50.412079 15844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:50.413479 15844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:50.413748 15844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:50.414016 15844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:50.419046 15850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:50.419061 15852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:50.419065 15849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:50.419322 15844 server_base.cc:1034] running on GCE node
I20260929 19:09:50.419639 15844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:50.419955 15844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:50.421130 15844 hybrid_clock.cc:648] HybridClock initialized: now 1790708990421112 us; error 45 us; skew 500 ppm
Sep 29 19:09:50 dist-test-slave-xz51 krb5kdc[15831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:50.423781 15844 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:50.424335 15844 webserver.cc:460] Webserver started at http://127.0.0.1:45219/ using document root <none> and password file <none>
I20260929 19:09:50.424598 15844 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:50.424672 15844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:50.424892 15844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:50.426147 15844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "8f7cebbfd28e438b84201770fca75cdd"
format_stamp: "Formatted at 2026-09-29 19:09:50 on dist-test-slave-xz51"
server_key: "5bf17732e6426998df8f08a232bcf64a"
server_key_iv: "a01fc233ef22e980380aa020c7175e7a"
server_key_version: "encryptionkey@0"
I20260929 19:09:50.426563 15844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "8f7cebbfd28e438b84201770fca75cdd"
format_stamp: "Formatted at 2026-09-29 19:09:50 on dist-test-slave-xz51"
server_key: "5bf17732e6426998df8f08a232bcf64a"
server_key_iv: "a01fc233ef22e980380aa020c7175e7a"
server_key_version: "encryptionkey@0"
I20260929 19:09:50.427994 15844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:50.428733 15859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:50.428943 15844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:50.429042 15844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "8f7cebbfd28e438b84201770fca75cdd"
format_stamp: "Formatted at 2026-09-29 19:09:50 on dist-test-slave-xz51"
server_key: "5bf17732e6426998df8f08a232bcf64a"
server_key_iv: "a01fc233ef22e980380aa020c7175e7a"
server_key_version: "encryptionkey@0"
I20260929 19:09:50.429134 15844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:50.442009 15844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:50.444629 15844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:50.444836 15844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:50.449139 15844 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33163
I20260929 19:09:50.449178 15911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33163 every 8 connection(s)
I20260929 19:09:50.449723 15844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:50.450445 15912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:50.453032 15912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd: Bootstrap starting.
I20260929 19:09:50.453679 15912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:50.453985 15912 log.cc:824] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:50.454263 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15844
I20260929 19:09:50.454389 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:50.454586 26939 external_mini_cluster.cc:1428] Setting key 71db5d18cc6843b2f5a522881896dc60
I20260929 19:09:50.454756 15912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd: No bootstrap required, opened a new log
I20260929 19:09:50.457782 15912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7cebbfd28e438b84201770fca75cdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260929 19:09:50.457986 15912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:50.458020 15912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7cebbfd28e438b84201770fca75cdd, State: Initialized, Role: FOLLOWER
I20260929 19:09:50.458137 15912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7cebbfd28e438b84201770fca75cdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260929 19:09:50.458230 15912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:50.458264 15912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:50.458312 15912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:50.458882 15912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7cebbfd28e438b84201770fca75cdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260929 19:09:50.459012 15912 leader_election.cc:302] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f7cebbfd28e438b84201770fca75cdd; no voters:
I20260929 19:09:50.459169 15912 leader_election.cc:288] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:50.459194 15917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:50.459380 15917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [term 1 LEADER]: Becoming Leader. State: Replica: 8f7cebbfd28e438b84201770fca75cdd, State: Running, Role: LEADER
I20260929 19:09:50.459472 15912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:50.459508 15917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7cebbfd28e438b84201770fca75cdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260929 19:09:50.460086 15919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f7cebbfd28e438b84201770fca75cdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7cebbfd28e438b84201770fca75cdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } } }
I20260929 19:09:50.460186 15919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:50.460000 15917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f7cebbfd28e438b84201770fca75cdd. Latest consensus state: current_term: 1 leader_uuid: "8f7cebbfd28e438b84201770fca75cdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7cebbfd28e438b84201770fca75cdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } } }
I20260929 19:09:50.460495 15917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:50.461416 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:50.455186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:60259 (local address 127.0.0.1:33163)
0929 19:09:50.455454 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:50.455464 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:50.455504 (+ 40us) server_negotiation.cc:407] Connection header received
0929 19:09:50.456210 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:50.456220 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:50.456287 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:50.456439 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:50.456642 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:50.458117 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:50.458975 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:50.459599 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:50.459776 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:50.460587 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:50.460592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:50.460593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:50.460805 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:50.460849 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:50.460867 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:50.460970 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:50.461179 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 19:09:50.461206 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":115,"threads_started":1}
W20260929 19:09:50.461444 15933 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:09:50.461519 15933 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:09:50.461581 15928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:50.461762 15928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:50.462366 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:50.463370 15928 catalog_manager.cc:1347] Generated new cluster ID: 1b6bdd5430d44b55aab294ffe711edd0
I20260929 19:09:50.463467 15928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:50.470083 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:50.466206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:37489 (local address 127.0.0.1:33163)
0929 19:09:50.466245 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:50.466247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:50.466418 (+ 171us) server_negotiation.cc:407] Connection header received
0929 19:09:50.466469 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:50.466470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:50.466481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:50.466539 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:50.466680 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:50.467492 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:50.468601 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:50.468924 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:50.468999 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:50.469809 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:50.469810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:50.469810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:50.469815 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:50.469851 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:50.469855 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:50.469913 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:50.469998 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 19:09:50.470008 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:09:50.476478 15928 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:50.477035 15928 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:50.490000 15928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f7cebbfd28e438b84201770fca75cdd: Generated new TSK 0
I20260929 19:09:50.490239 15928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:50.498481 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15844
W20260929 19:09:50.502502 15829 connection.cc:535] client connection to 127.0.0.1:33163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33163): Connection reset by peer (error 104)
2026-09-29T19:09:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:50 dist-test-slave-xz51 krb5kdc[15943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:50 dist-test-slave-xz51 krb5kdc[15943](info): set up 2 sockets
Sep 29 19:09:50 dist-test-slave-xz51 krb5kdc[15943](info): commencing operation
krb5kdc: starting...
W20260929 19:09:51.530138 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:51 dist-test-slave-xz51 krb5kdc[15943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:09:51.648629 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:51.730185 15956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:51.730453 15956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:51.730497 15956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:51.731676 15956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:51.731756 15956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:51.731781 15956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:51.731801 15956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:51.731830 15956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:51.733341 15956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:51.733660 15956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:51.733915 15956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:51.738780 15961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:51.738786 15964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:51.738828 15962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:51.738879 15956 server_base.cc:1034] running on GCE node
I20260929 19:09:51.739341 15956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:51.739637 15956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:51.740782 15956 hybrid_clock.cc:648] HybridClock initialized: now 1790708991740768 us; error 29 us; skew 500 ppm
Sep 29 19:09:51 dist-test-slave-xz51 krb5kdc[15943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:51.743320 15956 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:09:51.743808 15956 webserver.cc:460] Webserver started at http://127.0.0.1:43057/ using document root <none> and password file <none>
I20260929 19:09:51.744009 15956 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:51.744050 15956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:51.744138 15956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:51.745153 15956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7"
format_stamp: "Formatted at 2026-09-29 19:09:51 on dist-test-slave-xz51"
server_key: "40503f26762bba19bbcde84c8bbecb83"
server_key_iv: "5cb63747a34ad40a5f0bf0948378ec9d"
server_key_version: "encryptionkey@0"
I20260929 19:09:51.745446 15956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7"
format_stamp: "Formatted at 2026-09-29 19:09:51 on dist-test-slave-xz51"
server_key: "40503f26762bba19bbcde84c8bbecb83"
server_key_iv: "5cb63747a34ad40a5f0bf0948378ec9d"
server_key_version: "encryptionkey@0"
I20260929 19:09:51.746773 15956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:09:51.747480 15971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:51.747678 15956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:09:51.747761 15956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7"
format_stamp: "Formatted at 2026-09-29 19:09:51 on dist-test-slave-xz51"
server_key: "40503f26762bba19bbcde84c8bbecb83"
server_key_iv: "5cb63747a34ad40a5f0bf0948378ec9d"
server_key_version: "encryptionkey@0"
I20260929 19:09:51.747817 15956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:51.751127 15956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:51.753432 15956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:51.753602 15956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:51.757422 15956 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39741
I20260929 19:09:51.757474 16023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39741 every 8 connection(s)
I20260929 19:09:51.757805 15956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:51.758896 16024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:51.761154 16024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7: Bootstrap starting.
I20260929 19:09:51.761744 16024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:51.762085 16024 log.cc:824] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:51.762774 16024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7: No bootstrap required, opened a new log
I20260929 19:09:51.763834 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 15956
I20260929 19:09:51.763938 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:51.764145 26939 external_mini_cluster.cc:1428] Setting key 6a7a150c5c01903391e7c266a194e1a9
I20260929 19:09:51.764858 16024 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39741 } }
I20260929 19:09:51.765067 16024 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:51.765105 16024 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6784f366e845dbb3f65a1ef0fb6ee7, State: Initialized, Role: FOLLOWER
I20260929 19:09:51.765235 16024 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39741 } }
I20260929 19:09:51.765350 16024 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:51.765443 16024 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:51.765506 16024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:51.766525 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:51.764717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:42921 (local address 127.0.0.1:39741)
0929 19:09:51.764940 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:51.764950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:51.764983 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:09:51.765112 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:51.765121 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:51.765153 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:51.765295 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:51.766053 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:51.766059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:51.766059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:51.766065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:51.766121 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:51.766141 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:51.766222 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:51.766277 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 19:09:51.766321 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
I20260929 19:09:51.766697 16024 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39741 } }
I20260929 19:09:51.766815 16024 leader_election.cc:302] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff6784f366e845dbb3f65a1ef0fb6ee7; no voters:
I20260929 19:09:51.766979 16024 leader_election.cc:288] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:51.767014 16029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:51.767216 16024 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:51.767261 16029 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [term 1 LEADER]: Becoming Leader. State: Replica: ff6784f366e845dbb3f65a1ef0fb6ee7, State: Running, Role: LEADER
I20260929 19:09:51.767386 16029 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39741 } }
I20260929 19:09:51.767571 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:51.767791 16030 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39741 } } }
I20260929 19:09:51.767853 16030 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:51.768013 16031 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff6784f366e845dbb3f65a1ef0fb6ee7. Latest consensus state: current_term: 1 leader_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6784f366e845dbb3f65a1ef0fb6ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39741 } } }
I20260929 19:09:51.768066 16031 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7 [sys.catalog]: This master's current role is: LEADER
W20260929 19:09:51.769367 16046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:09:51.769425 16046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:09:51.769472 16036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:51.769610 16036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:51.770619 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:51.769295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:39545 (local address 127.0.0.1:39741)
0929 19:09:51.769333 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:51.769335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:51.769524 (+ 189us) server_negotiation.cc:407] Connection header received
0929 19:09:51.769614 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:51.769615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:51.769621 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:51.769670 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:51.770373 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:51.770374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:51.770375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:51.770379 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:51.770396 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:51.770400 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:51.770469 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:51.770538 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 19:09:51.770551 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 19:09:51.771224 16036 catalog_manager.cc:1347] Generated new cluster ID: 917fb93e16a34747b12692a639417a60
I20260929 19:09:51.771276 16036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:51.780120 16036 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:51.780308 16036 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:51.782056 16036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff6784f366e845dbb3f65a1ef0fb6ee7: Generated new TSK 0
I20260929 19:09:51.782255 16036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:51.784726 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 15956
2026-09-29T19:09:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1286 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:51 dist-test-slave-xz51 krb5kdc[16055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:51 dist-test-slave-xz51 krb5kdc[16055](info): set up 2 sockets
Sep 29 19:09:51 dist-test-slave-xz51 krb5kdc[16055](info): commencing operation
krb5kdc: starting...
W20260929 19:09:52.830114 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:52 dist-test-slave-xz51 krb5kdc[16055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:52.948828 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:34399
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:34399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:53.030946 16068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:53.031206 16068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:53.031245 16068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:53.032433 16068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:53.032509 16068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:53.032533 16068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:53.032557 16068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:53.032586 16068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:53.034155 16068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:34399
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:34399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:53.034448 16068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:53.034677 16068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:53.039467 16073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:53.039489 16076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:53.039500 16074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:53.039708 16068 server_base.cc:1034] running on GCE node
I20260929 19:09:53.039949 16068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:53.040270 16068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:53.041432 16068 hybrid_clock.cc:648] HybridClock initialized: now 1790708993041418 us; error 45 us; skew 500 ppm
Sep 29 19:09:53 dist-test-slave-xz51 krb5kdc[16055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:53.043946 16068 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:53.044425 16068 webserver.cc:460] Webserver started at http://127.26.78.254:46465/ using document root <none> and password file <none>
I20260929 19:09:53.044667 16068 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:53.044737 16068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:53.044875 16068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:53.046195 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "6c42d0631de54c32852140e6d76ee68a"
format_stamp: "Formatted at 2026-09-29 19:09:53 on dist-test-slave-xz51"
server_key: "9a9addbfb9a99b4be4a60822ea977306"
server_key_iv: "de603e7fff35f23ed619b724e317b73c"
server_key_version: "encryptionkey@0"
I20260929 19:09:53.046511 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "6c42d0631de54c32852140e6d76ee68a"
format_stamp: "Formatted at 2026-09-29 19:09:53 on dist-test-slave-xz51"
server_key: "9a9addbfb9a99b4be4a60822ea977306"
server_key_iv: "de603e7fff35f23ed619b724e317b73c"
server_key_version: "encryptionkey@0"
I20260929 19:09:53.047776 16068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:09:53.048475 16083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:53.048676 16068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:53.048772 16068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "6c42d0631de54c32852140e6d76ee68a"
format_stamp: "Formatted at 2026-09-29 19:09:53 on dist-test-slave-xz51"
server_key: "9a9addbfb9a99b4be4a60822ea977306"
server_key_iv: "de603e7fff35f23ed619b724e317b73c"
server_key_version: "encryptionkey@0"
I20260929 19:09:53.048849 16068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:53.080231 16068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:53.082212 16068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:53.082365 16068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:53.086364 16068 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:34399
I20260929 19:09:53.086440 16135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:34399 every 8 connection(s)
I20260929 19:09:53.086778 16068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:53.087615 16136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:53.089792 16136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a: Bootstrap starting.
I20260929 19:09:53.090375 16136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:53.090610 16136 log.cc:824] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:53.091223 16136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a: No bootstrap required, opened a new log
I20260929 19:09:53.092761 16136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42d0631de54c32852140e6d76ee68a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34399 } }
I20260929 19:09:53.092895 16136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:53.092918 16136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c42d0631de54c32852140e6d76ee68a, State: Initialized, Role: FOLLOWER
I20260929 19:09:53.093016 16136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42d0631de54c32852140e6d76ee68a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34399 } }
I20260929 19:09:53.093096 16136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:53.093133 16136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:53.093209 16136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:53.094040 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16068
I20260929 19:09:53.094156 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:53.094180 16136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42d0631de54c32852140e6d76ee68a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34399 } }
I20260929 19:09:53.094295 16136 leader_election.cc:302] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c42d0631de54c32852140e6d76ee68a; no voters:
I20260929 19:09:53.094365 26939 external_mini_cluster.cc:1428] Setting key b0b0f7959383b161ce8c2208c0bd592c
I20260929 19:09:53.094472 16136 leader_election.cc:288] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:53.094534 16139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:53.094717 16139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [term 1 LEADER]: Becoming Leader. State: Replica: 6c42d0631de54c32852140e6d76ee68a, State: Running, Role: LEADER
I20260929 19:09:53.094769 16136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:53.094849 16139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42d0631de54c32852140e6d76ee68a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34399 } }
I20260929 19:09:53.095582 16140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c42d0631de54c32852140e6d76ee68a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42d0631de54c32852140e6d76ee68a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34399 } } }
I20260929 19:09:53.095662 16140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:53.095825 16141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c42d0631de54c32852140e6d76ee68a. Latest consensus state: current_term: 1 leader_uuid: "6c42d0631de54c32852140e6d76ee68a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42d0631de54c32852140e6d76ee68a" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 34399 } } }
I20260929 19:09:53.095881 16141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:53.096211 16150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:53.097026 16150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:53.098521 16150 catalog_manager.cc:1347] Generated new cluster ID: bb11c790d5e44110a3bc10181430a947
I20260929 19:09:53.098577 16150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:09:53 dist-test-slave-xz51 krb5kdc[16055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:53.105574 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:53.094999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34695 (local address 127.26.78.254:34399)
0929 19:09:53.095144 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:53.095154 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:53.095189 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:09:53.095305 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:53.095312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:53.095367 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:53.096004 (+ 637us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:53.096352 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:53.098091 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:53.098967 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:53.099537 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:53.099649 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:53.101880 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:53.101883 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:53.101884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:53.101894 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:53.103832 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:53.104287 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:53.104289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:53.104290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:53.104404 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:53.104653 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:53.104654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:53.104654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:53.104949 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:53.105107 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:53.105243 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 19:09:53.105342 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260929 19:09:53.106590 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:53.115540 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:53.108127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47723 (local address 127.26.78.254:34399)
0929 19:09:53.108164 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:53.108166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:53.108257 (+ 91us) server_negotiation.cc:407] Connection header received
0929 19:09:53.108399 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:53.108400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:53.108410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:53.108486 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:53.109183 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:53.109979 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:53.110950 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:53.111269 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:53.111341 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:53.112690 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:53.112691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:53.112691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:53.112698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:53.114323 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:53.114724 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:53.114725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:53.114726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:53.114785 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:53.115013 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:53.115014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:53.115014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:53.115142 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:53.115241 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:53.115380 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 19:09:53.115443 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:09:53.130007 16150 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:53.130507 16150 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:53.135398 16150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c42d0631de54c32852140e6d76ee68a: Generated new TSK 0
I20260929 19:09:53.135596 16150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:53.138227 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16068
W20260929 19:09:53.142534 16053 connection.cc:535] client connection to 127.26.78.254:34399 recv error: Network error: failed to read from TLS socket (remote: 127.26.78.254:34399): Connection reset by peer (error 104)
2026-09-29T19:09:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:53 dist-test-slave-xz51 krb5kdc[16167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:53 dist-test-slave-xz51 krb5kdc[16167](info): set up 2 sockets
Sep 29 19:09:53 dist-test-slave-xz51 krb5kdc[16167](info): commencing operation
krb5kdc: starting...
W20260929 19:09:54.170081 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:54 dist-test-slave-xz51 krb5kdc[16167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:54.288162 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44893
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:54.368423 16180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:54.368637 16180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:54.368660 16180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:54.369863 16180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:54.369903 16180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:54.369917 16180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:54.369974 16180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:54.370002 16180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:54.371529 16180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:54.371793 16180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:54.372030 16180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:54.376864 16186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:54.377074 16185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:54.377090 16188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:54.377405 16180 server_base.cc:1034] running on GCE node
I20260929 19:09:54.377663 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:54.378018 16180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:54.379307 16180 hybrid_clock.cc:648] HybridClock initialized: now 1790708994379287 us; error 32 us; skew 500 ppm
Sep 29 19:09:54 dist-test-slave-xz51 krb5kdc[16167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708994, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:54.386858 16180 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:54.387365 16180 webserver.cc:460] Webserver started at http://127.26.78.254:44745/ using document root <none> and password file <none>
I20260929 19:09:54.387610 16180 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:54.387678 16180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:54.387832 16180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:54.388911 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "46b23fdc140f48ada6f31fa063c2c2e9"
format_stamp: "Formatted at 2026-09-29 19:09:54 on dist-test-slave-xz51"
server_key: "857c175eabbb2a6c039d02e395f72591"
server_key_iv: "22f4b2802e491ef5d9800869734275ad"
server_key_version: "encryptionkey@0"
I20260929 19:09:54.389281 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "46b23fdc140f48ada6f31fa063c2c2e9"
format_stamp: "Formatted at 2026-09-29 19:09:54 on dist-test-slave-xz51"
server_key: "857c175eabbb2a6c039d02e395f72591"
server_key_iv: "22f4b2802e491ef5d9800869734275ad"
server_key_version: "encryptionkey@0"
I20260929 19:09:54.390765 16180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:09:54.391534 16195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:54.391744 16180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:54.391852 16180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "46b23fdc140f48ada6f31fa063c2c2e9"
format_stamp: "Formatted at 2026-09-29 19:09:54 on dist-test-slave-xz51"
server_key: "857c175eabbb2a6c039d02e395f72591"
server_key_iv: "22f4b2802e491ef5d9800869734275ad"
server_key_version: "encryptionkey@0"
I20260929 19:09:54.391944 16180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:54.417933 16180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:54.420374 16180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:54.420550 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:54.424695 16180 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:44893
I20260929 19:09:54.424731 16247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:44893 every 8 connection(s)
I20260929 19:09:54.425118 16180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:54.426088 16248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:54.428319 16248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9: Bootstrap starting.
I20260929 19:09:54.428835 16248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:54.429071 16248 log.cc:824] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:54.429699 16248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9: No bootstrap required, opened a new log
I20260929 19:09:54.431269 16248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44893 } }
I20260929 19:09:54.431409 16248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:54.431432 16248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b23fdc140f48ada6f31fa063c2c2e9, State: Initialized, Role: FOLLOWER
I20260929 19:09:54.431550 16248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44893 } }
I20260929 19:09:54.431613 16248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:54.431634 16248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:54.431706 16248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:54.432508 16248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44893 } }
I20260929 19:09:54.432621 16248 leader_election.cc:302] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46b23fdc140f48ada6f31fa063c2c2e9; no voters:
I20260929 19:09:54.432796 16248 leader_election.cc:288] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:54.432835 16251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:54.433020 16251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [term 1 LEADER]: Becoming Leader. State: Replica: 46b23fdc140f48ada6f31fa063c2c2e9, State: Running, Role: LEADER
I20260929 19:09:54.433068 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16180
I20260929 19:09:54.433177 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:54.433061 16248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:54.433193 16251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44893 } }
I20260929 19:09:54.433396 26939 external_mini_cluster.cc:1428] Setting key af563d748191004629b728c9bfdd0fbb
I20260929 19:09:54.433575 16252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44893 } } }
I20260929 19:09:54.433653 16252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:54.433959 16253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46b23fdc140f48ada6f31fa063c2c2e9. Latest consensus state: current_term: 1 leader_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b23fdc140f48ada6f31fa063c2c2e9" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44893 } } }
I20260929 19:09:54.434028 16253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:54.434741 16258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:54.435266 16258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:54.436867 16258 catalog_manager.cc:1347] Generated new cluster ID: dc7bf17eb9dc4a669b58a25b6be73b58
I20260929 19:09:54.436920 16258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:54.439726 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:54.434279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33507 (local address 127.26.78.254:44893)
0929 19:09:54.434813 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:54.434824 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:54.434888 (+ 64us) server_negotiation.cc:407] Connection header received
0929 19:09:54.435019 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:54.435027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:54.435089 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:54.435245 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:54.435492 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:54.436905 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:54.437703 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:54.438238 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:54.438353 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:54.439117 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:54.439120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:54.439121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:54.439294 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:54.439324 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:54.439334 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:54.439428 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:54.439517 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 19:09:54.439546 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":391,"threads_started":1}
I20260929 19:09:54.442706 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:54.452893 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:54.446126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57255 (local address 127.26.78.254:44893)
0929 19:09:54.446163 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:54.446165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:54.450041 (+ 3876us) server_negotiation.cc:407] Connection header received
0929 19:09:54.450112 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:54.450113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:54.450126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:54.450202 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:54.450381 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:54.450911 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:54.451649 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:54.451870 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:54.451946 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:54.452627 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:54.452628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:54.452628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:54.452633 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:54.452648 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:54.452652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:54.452722 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:54.452788 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:09:54.452800 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:09:54.464469 16258 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:54.465056 16258 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:54.476832 16258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46b23fdc140f48ada6f31fa063c2c2e9: Generated new TSK 0
I20260929 19:09:54.477057 16258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:54.487917 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16180
W20260929 19:09:54.491974 16165 connection.cc:535] client connection to 127.26.78.254:44893 recv error: Network error: failed to read from TLS socket (remote: 127.26.78.254:44893): Connection reset by peer (error 104)
2026-09-29T19:09:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:54 dist-test-slave-xz51 krb5kdc[16279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:54 dist-test-slave-xz51 krb5kdc[16279](info): set up 2 sockets
Sep 29 19:09:54 dist-test-slave-xz51 krb5kdc[16279](info): commencing operation
krb5kdc: starting...
W20260929 19:09:55.510151 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:55 dist-test-slave-xz51 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:55.628698 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:37947
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:55.709748 16292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:55.709975 16292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:55.709998 16292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:55.711165 16292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:55.711210 16292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:55.711225 16292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:55.711275 16292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:55.711304 16292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:55.712812 16292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:37947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:37947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:55.713071 16292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:55.713307 16292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:55.718094 16297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:55.718127 16298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:55.718236 16292 server_base.cc:1034] running on GCE node
W20260929 19:09:55.718153 16300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:55.718499 16292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:55.718789 16292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:55.719956 16292 hybrid_clock.cc:648] HybridClock initialized: now 1790708995719949 us; error 40 us; skew 500 ppm
Sep 29 19:09:55 dist-test-slave-xz51 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708995, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:55.722345 16292 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:55.722813 16292 webserver.cc:460] Webserver started at http://127.26.78.254:40289/ using document root <none> and password file <none>
I20260929 19:09:55.723067 16292 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:55.723138 16292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:55.723284 16292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:55.724318 16292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "e143f689b4f24b44b1687f78ab19cdbb"
format_stamp: "Formatted at 2026-09-29 19:09:55 on dist-test-slave-xz51"
server_key: "08344fbeb2b811298d49eab220651cce"
server_key_iv: "6a071d623be7ab0373c293cf793c751b"
server_key_version: "encryptionkey@0"
I20260929 19:09:55.724668 16292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "e143f689b4f24b44b1687f78ab19cdbb"
format_stamp: "Formatted at 2026-09-29 19:09:55 on dist-test-slave-xz51"
server_key: "08344fbeb2b811298d49eab220651cce"
server_key_iv: "6a071d623be7ab0373c293cf793c751b"
server_key_version: "encryptionkey@0"
I20260929 19:09:55.725976 16292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:09:55.726766 16307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:55.726951 16292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:55.727052 16292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "e143f689b4f24b44b1687f78ab19cdbb"
format_stamp: "Formatted at 2026-09-29 19:09:55 on dist-test-slave-xz51"
server_key: "08344fbeb2b811298d49eab220651cce"
server_key_iv: "6a071d623be7ab0373c293cf793c751b"
server_key_version: "encryptionkey@0"
I20260929 19:09:55.727142 16292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:55.730422 16292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:55.732256 16292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:55.732415 16292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:55.736377 16292 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:37947
I20260929 19:09:55.736413 16359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:37947 every 8 connection(s)
I20260929 19:09:55.736735 16292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:55.737671 16360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:55.739914 16360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb: Bootstrap starting.
I20260929 19:09:55.740444 16360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:55.740679 16360 log.cc:824] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:55.741328 16360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb: No bootstrap required, opened a new log
I20260929 19:09:55.742992 16360 raft_consensus.cc:348] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e143f689b4f24b44b1687f78ab19cdbb" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37947 } }
I20260929 19:09:55.743127 16360 raft_consensus.cc:374] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:55.743108 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16292
I20260929 19:09:55.743150 16360 raft_consensus.cc:729] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e143f689b4f24b44b1687f78ab19cdbb, State: Initialized, Role: FOLLOWER
I20260929 19:09:55.743234 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:55.743316 16360 consensus_queue.cc:260] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e143f689b4f24b44b1687f78ab19cdbb" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37947 } }
I20260929 19:09:55.743402 16360 raft_consensus.cc:388] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:55.743424 16360 raft_consensus.cc:482] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:55.743472 26939 external_mini_cluster.cc:1428] Setting key 221e659498923b03a763c0980a4f36e4
I20260929 19:09:55.743495 16360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:55.744586 16360 raft_consensus.cc:504] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e143f689b4f24b44b1687f78ab19cdbb" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37947 } }
I20260929 19:09:55.744711 16360 leader_election.cc:302] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e143f689b4f24b44b1687f78ab19cdbb; no voters:
I20260929 19:09:55.744895 16360 leader_election.cc:288] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:55.745097 16366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:55.745182 16360 sys_catalog.cc:564] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:55.745271 16366 raft_consensus.cc:686] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [term 1 LEADER]: Becoming Leader. State: Replica: e143f689b4f24b44b1687f78ab19cdbb, State: Running, Role: LEADER
I20260929 19:09:55.745391 16366 consensus_queue.cc:237] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e143f689b4f24b44b1687f78ab19cdbb" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37947 } }
I20260929 19:09:55.745751 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:55.744014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47973 (local address 127.26.78.254:37947)
0929 19:09:55.744202 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:55.744210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:55.744236 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:09:55.744330 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:55.744335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:55.744358 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:55.744474 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:55.745212 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:55.745216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:55.745217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:55.745221 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:55.745266 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:55.745281 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:55.745321 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:55.745477 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 19:09:55.745521 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260929 19:09:55.745800 16365 sys_catalog.cc:455] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e143f689b4f24b44b1687f78ab19cdbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e143f689b4f24b44b1687f78ab19cdbb" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37947 } } }
I20260929 19:09:55.745908 16365 sys_catalog.cc:458] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:55.745812 16367 sys_catalog.cc:455] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e143f689b4f24b44b1687f78ab19cdbb. Latest consensus state: current_term: 1 leader_uuid: "e143f689b4f24b44b1687f78ab19cdbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e143f689b4f24b44b1687f78ab19cdbb" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 37947 } } }
I20260929 19:09:55.746104 16367 sys_catalog.cc:458] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:55.746202 16373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:55.746731 16373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:55.747016 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:55.749699 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:55.748631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35403 (local address 127.26.78.254:37947)
0929 19:09:55.748664 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:55.748666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:55.748831 (+ 165us) server_negotiation.cc:407] Connection header received
0929 19:09:55.748853 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:55.748854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:55.748858 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:55.748909 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:55.749499 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:55.749500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:55.749500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:55.749504 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:55.749518 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:55.749522 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:55.749590 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:55.749625 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 19:09:55.749638 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 19:09:55.750200 16373 catalog_manager.cc:1347] Generated new cluster ID: 8da8a8371f0c4b76bae2af1ba9f46960
I20260929 19:09:55.750252 16373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:55.756819 16373 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:55.756984 16373 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:55.764308 16373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e143f689b4f24b44b1687f78ab19cdbb: Generated new TSK 0
I20260929 19:09:55.764511 16373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:55.771027 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16292
2026-09-29T19:09:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1283 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:55 dist-test-slave-xz51 krb5kdc[16391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:55 dist-test-slave-xz51 krb5kdc[16391](info): set up 2 sockets
Sep 29 19:09:55 dist-test-slave-xz51 krb5kdc[16391](info): commencing operation
krb5kdc: starting...
W20260929 19:09:56.802207 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:56 dist-test-slave-xz51 krb5kdc[16391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:56.918691 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43003
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:43003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:56.998558 16404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:56.998806 16404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:56.998845 16404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:56.999987 16404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:57.000065 16404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:57.000090 16404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:57.000118 16404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:57.000145 16404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:57.001638 16404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:43003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:43003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:57.001981 16404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:57.002218 16404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:57.007181 16410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:57.007229 16409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:57.007227 16404 server_base.cc:1034] running on GCE node
W20260929 19:09:57.007190 16412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:57.007709 16404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:57.007970 16404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:57.009110 16404 hybrid_clock.cc:648] HybridClock initialized: now 1790708997009101 us; error 31 us; skew 500 ppm
Sep 29 19:09:57 dist-test-slave-xz51 krb5kdc[16391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708997, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:57.011639 16404 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:57.012108 16404 webserver.cc:460] Webserver started at http://127.26.78.254:46771/ using document root <none> and password file <none>
I20260929 19:09:57.012292 16404 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:57.012333 16404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:57.012430 16404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:57.013548 16404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "7d558cecbdc54f66b7adea93c74b477e"
format_stamp: "Formatted at 2026-09-29 19:09:57 on dist-test-slave-xz51"
server_key: "b9136ad8c658ed2cb7691397564ea1b4"
server_key_iv: "8a0178e1fa51616100bb2911400f767e"
server_key_version: "encryptionkey@0"
I20260929 19:09:57.013917 16404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "7d558cecbdc54f66b7adea93c74b477e"
format_stamp: "Formatted at 2026-09-29 19:09:57 on dist-test-slave-xz51"
server_key: "b9136ad8c658ed2cb7691397564ea1b4"
server_key_iv: "8a0178e1fa51616100bb2911400f767e"
server_key_version: "encryptionkey@0"
I20260929 19:09:57.015166 16404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:09:57.015825 16419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:57.015997 16404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:09:57.016088 16404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "7d558cecbdc54f66b7adea93c74b477e"
format_stamp: "Formatted at 2026-09-29 19:09:57 on dist-test-slave-xz51"
server_key: "b9136ad8c658ed2cb7691397564ea1b4"
server_key_iv: "8a0178e1fa51616100bb2911400f767e"
server_key_version: "encryptionkey@0"
I20260929 19:09:57.016165 16404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:57.031651 16404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:57.033519 16404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:57.033674 16404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:57.037949 16404 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:43003
I20260929 19:09:57.037998 16471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:43003 every 8 connection(s)
I20260929 19:09:57.038334 16404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:57.039131 16472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:57.041400 16472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e: Bootstrap starting.
I20260929 19:09:57.042054 16472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:57.042338 16472 log.cc:824] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:57.043138 16472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e: No bootstrap required, opened a new log
I20260929 19:09:57.043185 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16404
I20260929 19:09:57.043289 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:57.043505 26939 external_mini_cluster.cc:1428] Setting key 933940f2ec72c7069d4339bd7c648b9e
I20260929 19:09:57.045046 16472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d558cecbdc54f66b7adea93c74b477e" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43003 } }
I20260929 19:09:57.045210 16472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:57.045272 16472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d558cecbdc54f66b7adea93c74b477e, State: Initialized, Role: FOLLOWER
I20260929 19:09:57.045454 16472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d558cecbdc54f66b7adea93c74b477e" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43003 } }
I20260929 19:09:57.045552 16472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:57.045596 16472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:57.045645 16472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:57.046525 16472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d558cecbdc54f66b7adea93c74b477e" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43003 } }
I20260929 19:09:57.046645 16472 leader_election.cc:302] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d558cecbdc54f66b7adea93c74b477e; no voters:
I20260929 19:09:57.046821 16472 leader_election.cc:288] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:57.046864 16477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:57.047034 16477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [term 1 LEADER]: Becoming Leader. State: Replica: 7d558cecbdc54f66b7adea93c74b477e, State: Running, Role: LEADER
I20260929 19:09:57.047086 16472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:57.047175 16477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d558cecbdc54f66b7adea93c74b477e" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43003 } }
I20260929 19:09:57.047676 16478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d558cecbdc54f66b7adea93c74b477e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d558cecbdc54f66b7adea93c74b477e" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43003 } } }
I20260929 19:09:57.047767 16478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:57.047734 16479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d558cecbdc54f66b7adea93c74b477e. Latest consensus state: current_term: 1 leader_uuid: "7d558cecbdc54f66b7adea93c74b477e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d558cecbdc54f66b7adea93c74b477e" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 43003 } } }
I20260929 19:09:57.047935 16479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e [sys.catalog]: This master's current role is: LEADER
W20260929 19:09:57.049182 16492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:09:57.049336 16492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:09:57.049408 16484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:57.049602 16484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:09:57 dist-test-slave-xz51 krb5kdc[16391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:09:57.051865 16484 catalog_manager.cc:1347] Generated new cluster ID: 08d87406b4084deaab361db5bbfddc64
I20260929 19:09:57.051925 16484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:57.056116 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:57.044035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53167 (local address 127.26.78.254:43003)
0929 19:09:57.044228 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:57.044238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:57.044277 (+ 39us) server_negotiation.cc:407] Connection header received
0929 19:09:57.045030 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:57.045037 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:57.045084 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:57.045227 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:57.046002 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:57.047072 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:57.048247 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:57.048904 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:57.049020 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:57.051960 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:57.051964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:57.051964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:57.051976 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:57.054019 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:57.054409 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:57.054412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:57.054412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:57.054545 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:57.054747 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:57.054748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:57.054749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:57.055084 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:57.055673 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:57.055749 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 19:09:57.055909 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260929 19:09:57.057168 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:57.064271 16484 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:57.064813 16484 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:57.069337 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:57.058567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36107 (local address 127.26.78.254:43003)
0929 19:09:57.058600 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:57.058601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:57.062082 (+ 3481us) server_negotiation.cc:407] Connection header received
0929 19:09:57.062115 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:57.062116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:57.062131 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:57.062203 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:57.062801 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:57.063576 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:57.064528 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:57.064858 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:57.064934 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:57.066440 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:57.066442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:57.066442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:09:57.066449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:57.068116 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:57.068505 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:57.068506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:57.068506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:57.068560 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:09:57.068790 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:09:57.068791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:09:57.068791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:09:57.068921 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:57.069015 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:57.069165 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 19:09:57.069251 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 19:09:57.070554 16484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d558cecbdc54f66b7adea93c74b477e: Generated new TSK 0
I20260929 19:09:57.070724 16484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:57.077128 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16404
2026-09-29T19:09:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:57 dist-test-slave-xz51 krb5kdc[16503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:57 dist-test-slave-xz51 krb5kdc[16503](info): set up 2 sockets
Sep 29 19:09:57 dist-test-slave-xz51 krb5kdc[16503](info): commencing operation
krb5kdc: starting...
W20260929 19:09:58.106060 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:58 dist-test-slave-xz51 krb5kdc[16503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:58.224852 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38349
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:38349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:58.306727 16516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:58.306978 16516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:58.307015 16516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:58.308154 16516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:58.308230 16516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:58.308257 16516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:58.308310 16516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:58.308338 16516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:58.309806 16516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:38349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:38349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:58.310132 16516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:58.310359 16516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:58.315124 16522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:58.315140 16524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:58.315188 16521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:58.315304 16516 server_base.cc:1034] running on GCE node
I20260929 19:09:58.315766 16516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:58.316018 16516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:58.317170 16516 hybrid_clock.cc:648] HybridClock initialized: now 1790708998317150 us; error 36 us; skew 500 ppm
Sep 29 19:09:58 dist-test-slave-xz51 krb5kdc[16503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708998, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:58.319624 16516 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:58.320075 16516 webserver.cc:460] Webserver started at http://127.26.78.254:35091/ using document root <none> and password file <none>
I20260929 19:09:58.320300 16516 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:58.320344 16516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:58.320503 16516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:58.321549 16516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "85eb9ebbbc61474da395565723579ece"
format_stamp: "Formatted at 2026-09-29 19:09:58 on dist-test-slave-xz51"
server_key: "f7946b8998e76e56f5a9f9e49b14bd38"
server_key_iv: "2af919801338549fb2fa20c72716ef85"
server_key_version: "encryptionkey@0"
I20260929 19:09:58.321952 16516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "85eb9ebbbc61474da395565723579ece"
format_stamp: "Formatted at 2026-09-29 19:09:58 on dist-test-slave-xz51"
server_key: "f7946b8998e76e56f5a9f9e49b14bd38"
server_key_iv: "2af919801338549fb2fa20c72716ef85"
server_key_version: "encryptionkey@0"
I20260929 19:09:58.323246 16516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:58.323976 16531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:58.324168 16516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:58.324268 16516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "85eb9ebbbc61474da395565723579ece"
format_stamp: "Formatted at 2026-09-29 19:09:58 on dist-test-slave-xz51"
server_key: "f7946b8998e76e56f5a9f9e49b14bd38"
server_key_iv: "2af919801338549fb2fa20c72716ef85"
server_key_version: "encryptionkey@0"
I20260929 19:09:58.324342 16516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:58.332224 16516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:58.334018 16516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:58.334167 16516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:58.338260 16516 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:38349
I20260929 19:09:58.338289 16583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:38349 every 8 connection(s)
I20260929 19:09:58.338642 16516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:58.339458 16584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:58.339741 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16516
I20260929 19:09:58.339854 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:58.340082 26939 external_mini_cluster.cc:1428] Setting key ddbe41a3b2cd447cdf83d3ceb13e9712
I20260929 19:09:58.342751 16584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece: Bootstrap starting.
I20260929 19:09:58.343439 16584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:58.343741 16584 log.cc:824] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:58.344448 16584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece: No bootstrap required, opened a new log
I20260929 19:09:58.345558 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:58.340597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52955 (local address 127.26.78.254:38349)
0929 19:09:58.340755 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:58.340763 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:58.340791 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:09:58.341197 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:58.341203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:58.341251 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:58.341362 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:58.341608 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:58.342564 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:58.343644 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:58.344137 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:58.344221 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:58.344927 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:58.344931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:58.344931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:58.345084 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:58.345113 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:58.345123 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:58.345172 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:58.345264 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 19:09:58.345312 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260929 19:09:58.346639 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:58.346822 16584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eb9ebbbc61474da395565723579ece" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38349 } }
I20260929 19:09:58.346989 16584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:58.347019 16584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85eb9ebbbc61474da395565723579ece, State: Initialized, Role: FOLLOWER
I20260929 19:09:58.347146 16584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eb9ebbbc61474da395565723579ece" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38349 } }
I20260929 19:09:58.347221 16584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:58.347251 16584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:58.347294 16584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:58.348052 16584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eb9ebbbc61474da395565723579ece" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38349 } }
I20260929 19:09:58.348145 16584 leader_election.cc:302] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85eb9ebbbc61474da395565723579ece; no voters:
I20260929 19:09:58.348300 16584 leader_election.cc:288] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:58.348341 16593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:58.348522 16593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [term 1 LEADER]: Becoming Leader. State: Replica: 85eb9ebbbc61474da395565723579ece, State: Running, Role: LEADER
I20260929 19:09:58.348629 16584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:58.348642 16593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eb9ebbbc61474da395565723579ece" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38349 } }
I20260929 19:09:58.349078 16593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85eb9ebbbc61474da395565723579ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eb9ebbbc61474da395565723579ece" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38349 } } }
I20260929 19:09:58.349203 16593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:58.349385 16593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85eb9ebbbc61474da395565723579ece. Latest consensus state: current_term: 1 leader_uuid: "85eb9ebbbc61474da395565723579ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eb9ebbbc61474da395565723579ece" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 38349 } } }
I20260929 19:09:58.349472 16593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:58.349773 16603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:58.350293 16603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:58.351881 16603 catalog_manager.cc:1347] Generated new cluster ID: feaa48d05ba24aee9b5bd346077b1b4b
I20260929 19:09:58.351976 16603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:58.352110 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:58.348371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33073 (local address 127.26.78.254:38349)
0929 19:09:58.348434 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:58.348435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:58.348445 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:09:58.348505 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:58.348506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:58.348516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:58.348586 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:58.348769 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:58.349547 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:58.350749 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:09:58.350997 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:09:58.351080 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:09:58.351842 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:58.351843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:58.351844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:58.351848 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:58.351863 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:58.351868 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:58.351928 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:58.352021 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 19:09:58.352045 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:09:58.367036 16603 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:58.367781 16603 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:58.375195 16603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85eb9ebbbc61474da395565723579ece: Generated new TSK 0
I20260929 19:09:58.375406 16603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:58.383754 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16516
2026-09-29T19:09:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:58 dist-test-slave-xz51 krb5kdc[16615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:58 dist-test-slave-xz51 krb5kdc[16615](info): set up 2 sockets
Sep 29 19:09:58 dist-test-slave-xz51 krb5kdc[16615](info): commencing operation
krb5kdc: starting...
W20260929 19:09:59.442055 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:09:59 dist-test-slave-xz51 krb5kdc[16615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:09:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:09:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:09:59.560977 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:41171
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:41171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:09:59.641031 16628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:09:59.641278 16628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:09:59.641316 16628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:09:59.642477 16628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:09:59.642550 16628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:09:59.642575 16628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:09:59.642594 16628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:09:59.642622 16628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:09:59.644079 16628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:41171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:41171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:09:59.644351 16628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:09:59.644572 16628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:09:59.649385 16636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:59.649508 16634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:09:59.649392 16633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:09:59.649627 16628 server_base.cc:1034] running on GCE node
I20260929 19:09:59.649971 16628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:09:59.650282 16628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:09:59.651458 16628 hybrid_clock.cc:648] HybridClock initialized: now 1790708999651434 us; error 28 us; skew 500 ppm
Sep 29 19:09:59 dist-test-slave-xz51 krb5kdc[16615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790708999, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:09:59.659714 16628 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:09:59.660247 16628 webserver.cc:460] Webserver started at http://127.26.78.254:44573/ using document root <none> and password file <none>
I20260929 19:09:59.660473 16628 fs_manager.cc:359] Metadata directory not provided
I20260929 19:09:59.660538 16628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:09:59.660681 16628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:09:59.661751 16628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "ae0bb05f88e44179ae3adf95c034b0c0"
format_stamp: "Formatted at 2026-09-29 19:09:59 on dist-test-slave-xz51"
server_key: "0c06611460a64c369244b09dc2b51c1a"
server_key_iv: "fbeccd41c2b3b176f62d4357381dd630"
server_key_version: "encryptionkey@0"
I20260929 19:09:59.662178 16628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "ae0bb05f88e44179ae3adf95c034b0c0"
format_stamp: "Formatted at 2026-09-29 19:09:59 on dist-test-slave-xz51"
server_key: "0c06611460a64c369244b09dc2b51c1a"
server_key_iv: "fbeccd41c2b3b176f62d4357381dd630"
server_key_version: "encryptionkey@0"
I20260929 19:09:59.663475 16628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:09:59.664220 16643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:09:59.664386 16628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:09:59.664484 16628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "ae0bb05f88e44179ae3adf95c034b0c0"
format_stamp: "Formatted at 2026-09-29 19:09:59 on dist-test-slave-xz51"
server_key: "0c06611460a64c369244b09dc2b51c1a"
server_key_iv: "fbeccd41c2b3b176f62d4357381dd630"
server_key_version: "encryptionkey@0"
I20260929 19:09:59.664563 16628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:09:59.667897 16628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:09:59.670224 16628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:09:59.670399 16628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:09:59.674294 16628 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:41171
I20260929 19:09:59.674321 16695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:41171 every 8 connection(s)
I20260929 19:09:59.674646 16628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:09:59.675571 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16628
I20260929 19:09:59.675591 16696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:09:59.675681 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:09:59.675865 26939 external_mini_cluster.cc:1428] Setting key 262c4b3e4a8c661cb86e9ab7e89f3630
I20260929 19:09:59.678352 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:59.676369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37647 (local address 127.26.78.254:41171)
0929 19:09:59.676578 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:59.676587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:59.676622 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:09:59.677127 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:59.677136 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:59.677170 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:59.677317 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:59.677944 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:59.677948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:59.677948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:59.677952 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:59.678004 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:59.678015 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:59.678089 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:59.678157 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 19:09:59.678198 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260929 19:09:59.678961 16696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0: Bootstrap starting.
I20260929 19:09:59.679252 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:09:59.679647 16696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0: Neither blocks nor log segments found. Creating new log.
I20260929 19:09:59.679927 16696 log.cc:824] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:09:59.680725 16696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0: No bootstrap required, opened a new log
I20260929 19:09:59.681675 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:09:59.680667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48655 (local address 127.26.78.254:41171)
0929 19:09:59.680707 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:09:59.680709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:09:59.680829 (+ 120us) server_negotiation.cc:407] Connection header received
0929 19:09:59.680852 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:09:59.680853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:09:59.680858 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:09:59.680935 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:09:59.681424 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:09:59.681425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:09:59.681425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:09:59.681428 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:09:59.681439 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:09:59.681442 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:09:59.681503 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:09:59.681569 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:09:59.681578 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:09:59.683022 16696 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 41171 } }
I20260929 19:09:59.683156 16696 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:09:59.683179 16696 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0bb05f88e44179ae3adf95c034b0c0, State: Initialized, Role: FOLLOWER
I20260929 19:09:59.683326 16696 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 41171 } }
I20260929 19:09:59.683408 16696 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:09:59.683455 16696 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:09:59.683506 16696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:09:59.684036 16696 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 41171 } }
I20260929 19:09:59.684149 16696 leader_election.cc:302] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae0bb05f88e44179ae3adf95c034b0c0; no voters:
I20260929 19:09:59.684311 16696 leader_election.cc:288] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:09:59.684361 16706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:09:59.684536 16706 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [term 1 LEADER]: Becoming Leader. State: Replica: ae0bb05f88e44179ae3adf95c034b0c0, State: Running, Role: LEADER
I20260929 19:09:59.684576 16696 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:09:59.684649 16706 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 41171 } }
I20260929 19:09:59.685184 16707 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 41171 } } }
I20260929 19:09:59.685287 16707 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:59.685678 16708 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae0bb05f88e44179ae3adf95c034b0c0. Latest consensus state: current_term: 1 leader_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0bb05f88e44179ae3adf95c034b0c0" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 41171 } } }
I20260929 19:09:59.685745 16708 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:09:59.686067 16715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:09:59.686695 16715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:09:59.688309 16715 catalog_manager.cc:1347] Generated new cluster ID: 53ca0231d9224d118892659784fff3e5
I20260929 19:09:59.688364 16715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:09:59.707882 16715 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:09:59.708087 16715 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:09:59.712930 16715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae0bb05f88e44179ae3adf95c034b0c0: Generated new TSK 0
I20260929 19:09:59.713121 16715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:09:59.716979 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16628
2026-09-29T19:09:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1333 ms)
[----------] 18 tests from AuthTokenIssuingTest (23899 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:09:59 dist-test-slave-xz51 krb5kdc[16727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:09:59 dist-test-slave-xz51 krb5kdc[16727](info): set up 2 sockets
Sep 29 19:09:59 dist-test-slave-xz51 krb5kdc[16727](info): commencing operation
krb5kdc: starting...
W20260929 19:10:00.746085 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:10:00 dist-test-slave-xz51 krb5kdc[16727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:10:00.864782 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44739
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:32923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44739,127.26.78.253:36345,127.26.78.252:37239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:00.945473 16740 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:10:00.945736 16740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:00.945775 16740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:00.945814 16740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:00.946961 16740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:00.947031 16740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:00.947055 16740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:00.947085 16740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:00.947111 16740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:00.948589 16740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:32923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44739,127.26.78.253:36345,127.26.78.252:37239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:00.948868 16740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:00.949092 16740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:00.954131 16748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:00.954242 16746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:00.954159 16740 server_base.cc:1034] running on GCE node
W20260929 19:10:00.954131 16745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:00.954598 16740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:00.954905 16740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:00.956067 16740 hybrid_clock.cc:648] HybridClock initialized: now 1790709000956056 us; error 39 us; skew 500 ppm
Sep 29 19:10:00 dist-test-slave-xz51 krb5kdc[16727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:00.958523 16740 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:10:00.959013 16740 webserver.cc:460] Webserver started at http://127.26.78.254:35239/ using document root <none> and password file <none>
I20260929 19:10:00.959249 16740 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:00.959312 16740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:00.959461 16740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:00.960515 16740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "625f19e6ff7c4659b9189d17555fe7a5"
format_stamp: "Formatted at 2026-09-29 19:10:00 on dist-test-slave-xz51"
server_key: "4abefbe047eea0b8cde6242cc3f1b98f"
server_key_iv: "daee3eb591fcd6be854a44a1324322e5"
server_key_version: "encryptionkey@0"
I20260929 19:10:00.960856 16740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "625f19e6ff7c4659b9189d17555fe7a5"
format_stamp: "Formatted at 2026-09-29 19:10:00 on dist-test-slave-xz51"
server_key: "4abefbe047eea0b8cde6242cc3f1b98f"
server_key_iv: "daee3eb591fcd6be854a44a1324322e5"
server_key_version: "encryptionkey@0"
I20260929 19:10:00.962159 16740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:00.962872 16755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:00.963070 16740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:00.963166 16740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "625f19e6ff7c4659b9189d17555fe7a5"
format_stamp: "Formatted at 2026-09-29 19:10:00 on dist-test-slave-xz51"
server_key: "4abefbe047eea0b8cde6242cc3f1b98f"
server_key_iv: "daee3eb591fcd6be854a44a1324322e5"
server_key_version: "encryptionkey@0"
I20260929 19:10:00.963243 16740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:00.981238 16740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:00.983395 16740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:00.983554 16740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:00.987421 16740 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:44739
I20260929 19:10:00.987527 16807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:44739 every 8 connection(s)
I20260929 19:10:00.987828 16740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:00.988858 16808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:00.989399 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16740
I20260929 19:10:00.989482 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:10:00.989661 26939 external_mini_cluster.cc:1428] Setting key 6094d1ca6dc48a92e7cc0e06e9db93a5
I20260929 19:10:00.990626 16808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.78.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.253@KRBTEST.COM" created.
Sep 29 19:10:00 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:10:01.003818 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:00.991523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44739 (local address 127.0.0.1:58496)
0929 19:10:00.991781 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:00.991810 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:00.992270 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:00.992551 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:00.992553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:00.992833 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:00.993034 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:00.993039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:00.994110 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:00.994110 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:00.994778 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:00.994780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:00.995312 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:00.995313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:00.995378 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:00.995911 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:00.995919 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.000065 (+ 4146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.002292 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.002299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.002300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.002523 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.002803 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.002804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.002805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.002911 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.003368 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.003373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.003432 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.003507 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.003637 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
I20260929 19:10:01.003922 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:00.991542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.26.78.254:44739)
0929 19:10:00.991704 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:00.991714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:00.991901 (+ 187us) server_negotiation.cc:407] Connection header received
0929 19:10:00.992422 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:00.992424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:00.992443 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:00.992543 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:00.993134 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:00.994011 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:00.994876 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:00.995193 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:00.995335 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.000159 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.000166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.000167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.000173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.002205 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.002607 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.002611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.002611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.002722 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.002979 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.002980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.002981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.003298 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.003390 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.003570 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.003729 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260929 19:10:01.004739 16808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:01.005086 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.004977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38784)
0929 19:10:01.005022 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.005067 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 19:10:01.005240 16758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
I20260929 19:10:01.005686 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.005598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38794)
0929 19:10:01.005641 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.005665 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 19:10:01.005805 16808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:36345: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab.
Entry for principal kudu/127.26.78.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab.
I20260929 19:10:01.016033 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:36345
--webserver_interface=127.26.78.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab
--principal=kudu/127.26.78.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:32923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44739,127.26.78.253:36345,127.26.78.252:37239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:01.055058 16808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } attempt: 1
I20260929 19:10:01.055579 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.055440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38804)
0929 19:10:01.055503 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.055547 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 19:10:01.056066 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.055968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38818)
0929 19:10:01.056017 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.056046 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 19:10:01.056167 16808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:36345: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
W20260929 19:10:01.096722 16815 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:10:01.096961 16815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:01.097014 16815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:01.097050 16815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:01.098205 16815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:01.098259 16815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:01.098284 16815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:01.098313 16815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:01.098338 16815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:01.099784 16815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:32923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44739,127.26.78.253:36345,127.26.78.252:37239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab
--principal=kudu/127.26.78.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:36345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.78.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:01.100052 16815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:01.100277 16815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:01.105039 16823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:01.105113 16820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:01.105064 16821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:01.105266 16815 server_base.cc:1034] running on GCE node
I20260929 19:10:01.105520 16815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:01.105800 16815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:01.107002 16815 hybrid_clock.cc:648] HybridClock initialized: now 1790709001106981 us; error 37 us; skew 500 ppm
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:01.109619 16815 init.cc:377] Logged in from keytab as kudu/127.26.78.253@KRBTEST.COM (short username kudu)
I20260929 19:10:01.110082 16815 webserver.cc:460] Webserver started at http://127.26.78.253:40767/ using document root <none> and password file <none>
I20260929 19:10:01.110296 16815 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:01.110339 16815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:01.110515 16815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:01.111578 16815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data/instance:
uuid: "c5cfb79f954141f6ad2e78bac19f87ad"
format_stamp: "Formatted at 2026-09-29 19:10:01 on dist-test-slave-xz51"
server_key: "7b3d8fdec6a362ff4ba1c4f27d46c6d2"
server_key_iv: "d8b17b0382b5b5ea46805843150ac0fc"
server_key_version: "encryptionkey@0"
I20260929 19:10:01.111922 16815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/wal/instance:
uuid: "c5cfb79f954141f6ad2e78bac19f87ad"
format_stamp: "Formatted at 2026-09-29 19:10:01 on dist-test-slave-xz51"
server_key: "7b3d8fdec6a362ff4ba1c4f27d46c6d2"
server_key_iv: "d8b17b0382b5b5ea46805843150ac0fc"
server_key_version: "encryptionkey@0"
I20260929 19:10:01.113229 16815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:01.114034 16830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:01.114185 16815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:01.114286 16815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/wal
uuid: "c5cfb79f954141f6ad2e78bac19f87ad"
format_stamp: "Formatted at 2026-09-29 19:10:01 on dist-test-slave-xz51"
server_key: "7b3d8fdec6a362ff4ba1c4f27d46c6d2"
server_key_iv: "d8b17b0382b5b5ea46805843150ac0fc"
server_key_version: "encryptionkey@0"
I20260929 19:10:01.114355 16815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:01.129212 16808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } attempt: 2
I20260929 19:10:01.129717 16815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:01.130686 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.130557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38824)
0929 19:10:01.130615 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.130643 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 19:10:01.131109 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.131013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38832)
0929 19:10:01.131048 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.131078 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260929 19:10:01.131280 16808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:36345: Network error: Client connection negotiation failed: client connection to 127.26.78.253:36345: connect: Connection refused (error 111)
I20260929 19:10:01.133924 16815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:01.134081 16815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:01.138531 16815 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.253:36345
I20260929 19:10:01.138566 16882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.253:36345 every 8 connection(s)
I20260929 19:10:01.139019 16815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
I20260929 19:10:01.140276 16883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:01.141073 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16815
I20260929 19:10:01.141178 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-1/wal/instance
I20260929 19:10:01.141366 26939 external_mini_cluster.cc:1428] Setting key 5117a5f4ec8948d5618beed8576cecf8
I20260929 19:10:01.141983 16883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.78.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.252@KRBTEST.COM" created.
I20260929 19:10:01.151980 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.142718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.26.78.254:44739)
0929 19:10:01.142775 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.142778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.143058 (+ 280us) server_negotiation.cc:407] Connection header received
0929 19:10:01.143747 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.143749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.143766 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.143829 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.144469 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.144957 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.146553 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.146763 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.146831 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.149255 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.149257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.149257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.149266 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.150686 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.151045 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.151046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.151046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.151096 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.151367 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.151367 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.151368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.151511 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.151566 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.151797 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.151881 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:10:01.152227 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.142709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44739 (local address 127.0.0.1:58498)
0929 19:10:01.142906 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.142959 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.143645 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.143901 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.143905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.144167 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.144363 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.144367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.145051 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.145052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.146464 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.146468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.146872 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.146874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.147028 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.147598 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.147610 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.149140 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.150754 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.150760 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.150762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.150977 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.151173 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.151174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.151174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.151309 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.151570 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.151575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.151688 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.151770 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.151956 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260929 19:10:01.152766 16883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:10:01.160359 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.153032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38844)
0929 19:10:01.153206 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.153215 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.153250 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.153395 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.153396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.153548 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.153612 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.153614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.154495 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.154496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.155360 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.155362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.155875 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.155876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.155964 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.156311 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.156316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.157592 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.159348 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.159348 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.159349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.159510 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.159711 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.159712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.159712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.159764 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.160054 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.160055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.160093 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.160149 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.160247 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260929 19:10:01.160431 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.153090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.26.78.253:36345)
0929 19:10:01.153249 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.153255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.153281 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:10:01.153304 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.153306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.153321 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.153390 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.153693 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.154412 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.155447 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.155798 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.155853 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.157682 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.157694 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.157694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.157699 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.159254 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.159595 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.159598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.159599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.159651 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.159842 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.159843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.159843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.159978 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.160064 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.160187 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.160298 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260929 19:10:01.161087 16883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:01.161440 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.161336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36036)
0929 19:10:01.161367 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.161409 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260929 19:10:01.161550 16833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
I20260929 19:10:01.162011 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.161915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36040)
0929 19:10:01.161967 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.161992 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 19:10:01.162132 16883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:37239: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab.
Entry for principal kudu/127.26.78.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab.
I20260929 19:10:01.169158 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:37239
--webserver_interface=127.26.78.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab
--principal=kudu/127.26.78.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:32923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44739,127.26.78.253:36345,127.26.78.252:37239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:01.211362 16883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } attempt: 1
I20260929 19:10:01.211934 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.211786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36048)
0929 19:10:01.211850 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.211897 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 19:10:01.212337 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.212233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36052)
0929 19:10:01.212280 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.212312 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 19:10:01.212452 16883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:37239: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
I20260929 19:10:01.222417 16808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } attempt: 3
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:10:01.229405 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.222807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.26.78.253:36345)
0929 19:10:01.222853 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.222857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.222899 (+ 42us) server_negotiation.cc:407] Connection header received
0929 19:10:01.222981 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.222982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.222996 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.223057 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.223382 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.223869 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.224548 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.224814 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.224890 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.226855 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.226857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.226858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.226866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.228267 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.228627 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.228628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.228629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.228695 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.228890 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.228891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.228892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.229010 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.229082 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.229229 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.229313 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:10:01.229405 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.222740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38856)
0929 19:10:01.222846 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.222858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.222918 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.223062 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.223063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.223237 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.223301 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.223304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.223948 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.223949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.224467 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.224469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.224889 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.224891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.224985 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.225486 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.225496 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.226764 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.228374 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.228375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.228376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.228546 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.228762 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.228763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.228763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.228831 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.229075 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.229076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.229124 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.229205 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.229307 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 19:10:01.229751 16808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:01.230115 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.230020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36064)
0929 19:10:01.230066 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.230092 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 19:10:01.230397 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.230317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36074)
0929 19:10:01.230356 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.230379 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 19:10:01.230479 16808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:37239: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
W20260929 19:10:01.250425 16890 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:10:01.250656 16890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:01.250686 16890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:01.250738 16890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:01.251890 16890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:01.251957 16890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:01.251983 16890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:01.252014 16890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:01.252055 16890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:01.253506 16890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:32923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44739,127.26.78.253:36345,127.26.78.252:37239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab
--principal=kudu/127.26.78.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:37239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.78.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:01.253765 16890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:01.254009 16890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:01.258876 16898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:01.258885 16896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:01.258888 16895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:01.259138 16890 server_base.cc:1034] running on GCE node
I20260929 19:10:01.259429 16890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:01.259703 16890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:01.260833 16890 hybrid_clock.cc:648] HybridClock initialized: now 1790709001260822 us; error 29 us; skew 500 ppm
I20260929 19:10:01.261610 16808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } attempt: 1
I20260929 19:10:01.262029 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.261902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36084)
0929 19:10:01.261967 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.261999 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 19:10:01.262408 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.262330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36096)
0929 19:10:01.262370 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.262393 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 19:10:01.262518 16808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:37239: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:01.263525 16890 init.cc:377] Logged in from keytab as kudu/127.26.78.252@KRBTEST.COM (short username kudu)
I20260929 19:10:01.263990 16890 webserver.cc:460] Webserver started at http://127.26.78.252:37571/ using document root <none> and password file <none>
I20260929 19:10:01.264211 16890 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:01.264259 16890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:01.264421 16890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:01.265478 16890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data/instance:
uuid: "a865839a9e62463e92ec17ad7fea26c2"
format_stamp: "Formatted at 2026-09-29 19:10:01 on dist-test-slave-xz51"
server_key: "19889d5ccf13903e630318536efd9ed5"
server_key_iv: "2a6e9ef619242360bc390d57097d13f8"
server_key_version: "encryptionkey@0"
I20260929 19:10:01.265908 16890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/wal/instance:
uuid: "a865839a9e62463e92ec17ad7fea26c2"
format_stamp: "Formatted at 2026-09-29 19:10:01 on dist-test-slave-xz51"
server_key: "19889d5ccf13903e630318536efd9ed5"
server_key_iv: "2a6e9ef619242360bc390d57097d13f8"
server_key_version: "encryptionkey@0"
I20260929 19:10:01.267215 16890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:01.267928 16905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:01.268110 16890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:01.268208 16890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/wal
uuid: "a865839a9e62463e92ec17ad7fea26c2"
format_stamp: "Formatted at 2026-09-29 19:10:01 on dist-test-slave-xz51"
server_key: "19889d5ccf13903e630318536efd9ed5"
server_key_iv: "2a6e9ef619242360bc390d57097d13f8"
server_key_version: "encryptionkey@0"
I20260929 19:10:01.268289 16890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:01.280622 16883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } attempt: 2
I20260929 19:10:01.281155 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.281005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36112)
0929 19:10:01.281080 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.281123 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 19:10:01.281590 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.281493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36124)
0929 19:10:01.281535 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.281563 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 19:10:01.281731 16883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:37239: Network error: Client connection negotiation failed: client connection to 127.26.78.252:37239: connect: Connection refused (error 111)
I20260929 19:10:01.283502 16890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:01.285450 16890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:01.285597 16890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:01.289651 16890 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.252:37239
I20260929 19:10:01.289721 16957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.252:37239 every 8 connection(s)
I20260929 19:10:01.290088 16890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
I20260929 19:10:01.291051 16958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:01.292589 16958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:01.293732 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 16890
I20260929 19:10:01.293861 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790708710206825-26939-0/minicluster-data/master-2/wal/instance
I20260929 19:10:01.294056 26939 external_mini_cluster.cc:1428] Setting key 33a2b776e539ba144929327944d7b4ff
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:10:01.302120 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.293335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58508 (local address 127.26.78.254:44739)
0929 19:10:01.293391 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.293393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.293562 (+ 169us) server_negotiation.cc:407] Connection header received
0929 19:10:01.294124 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.294125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.294138 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.294223 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.294991 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.295504 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.296389 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.296620 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.296672 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.299156 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.299158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.299158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.299167 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.300665 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.301104 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.301105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.301106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.301166 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.301446 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.301447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.301447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.301560 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.301632 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.301927 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.302012 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:10:01.302352 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.293263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44739 (local address 127.0.0.1:58508)
0929 19:10:01.293434 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.293475 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.294017 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.294322 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.294327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.294679 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.294864 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.294869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.295586 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.295587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.296309 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.296311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.296716 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.296718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.296846 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.297453 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.297461 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.299064 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.300741 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.300748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.300750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.301006 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.301239 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.301239 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.301240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.301354 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.301625 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.301631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.301769 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.301894 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.302109 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260929 19:10:01.302891 16958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:01.303313 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.294481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36134 (local address 127.26.78.252:37239)
0929 19:10:01.294820 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.294823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.294838 (+ 15us) server_negotiation.cc:407] Connection header received
0929 19:10:01.294854 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.294855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.294870 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.294926 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.295529 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.296333 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.297277 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.297646 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.297719 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.300431 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.300437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.300437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.300443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.302089 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.302425 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.302429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.302429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.302494 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.302717 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.302718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.302718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.302877 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.303002 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.303112 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.303197 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":97,"threads_started":1}
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:10:01.309912 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.303214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.26.78.253:36345)
0929 19:10:01.303268 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.303270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.303279 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:01.303305 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.303306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.303317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.303372 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.303732 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.304359 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.305089 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.305366 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.305416 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.307364 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.307365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.307366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.307374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.308773 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.309129 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.309130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.309131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.309194 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.309409 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.309410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.309410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.309513 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.309565 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.309718 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.309776 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:10:01.309942 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.303142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:36345 (local address 127.0.0.1:38858)
0929 19:10:01.303186 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.303195 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.303226 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.303491 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.303492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.303612 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.303654 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.303655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.304435 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.304436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.305011 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.305014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.305433 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.305434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.305503 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.306047 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.306058 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.307264 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.308847 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.308848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.308849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.309059 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.309259 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.309260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.309260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.309335 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.309578 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.309579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.309620 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.309694 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.309781 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 19:10:01.310432 16958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:01.311195 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.304042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.26.78.254:44739)
0929 19:10:01.304081 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.304083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.304090 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:01.304109 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.304109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.304117 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.304178 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.304666 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.305161 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.305996 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.306203 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.306261 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.308328 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.308329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.308329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.308336 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.310082 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.310462 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.310463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.310464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.310518 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.310702 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.310703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.310703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.310813 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.310970 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.311007 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.311083 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:10:01.317356 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.310670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36148)
0929 19:10:01.310715 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.310725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.310768 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.310881 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.310882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.311014 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.311036 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.311038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.311870 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.311870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.312685 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.312688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.313137 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.313137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.313172 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.313542 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.313546 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.314856 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.316341 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.316341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.316342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.316500 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.316689 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.316690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.316691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.316742 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.317026 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.317026 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.317060 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.317139 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.317253 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 19:10:01.317356 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.310702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.26.78.252:37239)
0929 19:10:01.310733 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.310734 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.310755 (+ 21us) server_negotiation.cc:407] Connection header received
0929 19:10:01.310821 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.310822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.310831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.310870 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.311102 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.311794 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.312755 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.313063 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.313124 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.314933 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.314934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.314934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.314940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.316233 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.316567 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.316568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.316568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.316620 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.316836 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.316837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.316838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.316939 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.317010 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.317160 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.317276 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":17}
I20260929 19:10:01.318773 16958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: Bootstrap starting.
I20260929 19:10:01.319617 16958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:01.319866 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.312035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.26.78.253:36345)
0929 19:10:01.312067 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.312068 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.312076 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:01.312113 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.312114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.312126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.312166 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.312680 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.313343 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.314265 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.314597 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.314661 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.316987 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.316988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.316988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.316995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.318759 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.319101 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.319102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.319103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.319159 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.319345 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.319345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.319346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.319458 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.319641 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.319676 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.319749 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 19:10:01.319944 16958 log.cc:824] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:01.320735 16958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: No bootstrap required, opened a new log
I20260929 19:10:01.321306 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:10:01.322604 16958 raft_consensus.cc:348] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
I20260929 19:10:01.322732 16958 raft_consensus.cc:729] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a865839a9e62463e92ec17ad7fea26c2, State: Initialized, Role: FOLLOWER
I20260929 19:10:01.322814 16958 consensus_queue.cc:260] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
I20260929 19:10:01.322309 16782 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "625f19e6ff7c4659b9189d17555fe7a5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58514
I20260929 19:10:01.323135 16958 sys_catalog.cc:564] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:01.323118 16964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.323289 16964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:10:01.324790 16975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:01.324862 16975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:01.325088 16890 master_runner.cc:186] Error getting master registration for 127.26.78.254:44739: OK, instance_id { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" instance_seqno: 1790709000986306 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:10:01.326753 16890 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:10:01.337649 16808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } attempt: 2
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709000, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:10:01.345420 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.338040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36156)
0929 19:10:01.338091 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.338104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.338178 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.338322 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.338323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.338495 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.338542 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.338543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.339249 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.339250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.339757 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.339759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.340133 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.340134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.340196 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.340694 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.340699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.342340 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.344229 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.344230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.344231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.344474 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.344675 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.344676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.344676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.344745 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.345066 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.345067 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.345121 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.345198 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.345310 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 19:10:01.345420 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.338113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.26.78.252:37239)
0929 19:10:01.338181 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.338183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.338194 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:01.338249 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.338250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.338260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.338316 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.338640 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.339149 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.339845 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.340051 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.340123 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.342436 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.342437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.342437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.342445 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.344135 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.344547 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.344548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.344549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.344609 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.344811 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.344812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.344812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.344997 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.345065 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.345227 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.345311 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:10:01.347190 16808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5: Bootstrap starting.
I20260929 19:10:01.348038 16808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:01.348356 16808 log.cc:824] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:01.349256 16808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5: No bootstrap required, opened a new log
I20260929 19:10:01.350369 16808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
I20260929 19:10:01.350514 16808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625f19e6ff7c4659b9189d17555fe7a5, State: Initialized, Role: FOLLOWER
I20260929 19:10:01.350610 16808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
I20260929 19:10:01.350900 16978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.350952 16808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:01.350983 16978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:01.351104 16782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 19:10:01.351230 16782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:01.352118 16782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
W20260929 19:10:01.352969 16989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:01.353034 16989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:01.353101 16932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "625f19e6ff7c4659b9189d17555fe7a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a865839a9e62463e92ec17ad7fea26c2"
I20260929 19:10:01.353236 16932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:01.354163 16932 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625f19e6ff7c4659b9189d17555fe7a5 in term 1.
I20260929 19:10:01.354586 16758 leader_election.cc:302] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 625f19e6ff7c4659b9189d17555fe7a5, a865839a9e62463e92ec17ad7fea26c2; no voters:
I20260929 19:10:01.354723 16978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:01.354846 16978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [term 1 LEADER]: Becoming Leader. State: Replica: 625f19e6ff7c4659b9189d17555fe7a5, State: Running, Role: LEADER
I20260929 19:10:01.354925 16978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
I20260929 19:10:01.355326 16978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625f19e6ff7c4659b9189d17555fe7a5. Latest consensus state: current_term: 1 leader_uuid: "625f19e6ff7c4659b9189d17555fe7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.355406 16978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:01.355715 16782 leader_election.cc:288] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [CANDIDATE]: Term 1 election: Requested vote from peers c5cfb79f954141f6ad2e78bac19f87ad (127.26.78.253:36345), a865839a9e62463e92ec17ad7fea26c2 (127.26.78.252:37239)
I20260929 19:10:01.354315 16740 master_runner.cc:186] Error getting master registration for 127.26.78.253:36345: OK, instance_id { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" instance_seqno: 1790709001137240 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:10:01.356606 16740 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:10:01.362010 16993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:01.362244 16993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:01.365365 16932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [term 1 FOLLOWER]: Refusing update from remote peer 625f19e6ff7c4659b9189d17555fe7a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:01.365813 16991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:01.367300 16964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625f19e6ff7c4659b9189d17555fe7a5. Latest consensus state: current_term: 1 leader_uuid: "625f19e6ff7c4659b9189d17555fe7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.367389 16964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:01.369311 16991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "625f19e6ff7c4659b9189d17555fe7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.369400 16991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:01.369868 16964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "625f19e6ff7c4659b9189d17555fe7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.369953 16964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:01.370445 16999 mvcc.cc:205] Tried to move back new op lower bound from 7334744069589417984 to 7334744069550407680. Current Snapshot: MvccSnapshot[applied={T|T < 7334744069589417984}]
I20260929 19:10:01.371102 17002 mvcc.cc:205] Tried to move back new op lower bound from 7334744069589417984 to 7334744069550407680. Current Snapshot: MvccSnapshot[applied={T|T < 7334744069589417984}]
I20260929 19:10:01.371869 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.362955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.26.78.252:37239)
0929 19:10:01.362995 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.362997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.363589 (+ 592us) server_negotiation.cc:407] Connection header received
0929 19:10:01.363690 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.363691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.363701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.363731 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.364460 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.365154 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.366339 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.366666 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.366707 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.368507 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.368508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.368509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.368516 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.370509 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.370930 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.370932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.370932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.370990 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.371240 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.371241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.371241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.371377 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.371467 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.371687 (+ 220us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.371763 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:10:01.372885 16883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } attempt: 3
I20260929 19:10:01.373052 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.362703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38878 (local address 127.26.78.253:36345)
0929 19:10:01.362755 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.362758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.362951 (+ 193us) server_negotiation.cc:407] Connection header received
0929 19:10:01.363157 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.363158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.363171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.363205 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.364338 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.365219 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.366647 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.366950 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.366995 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.369209 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.369210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.369211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.369230 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.371011 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.371681 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.371682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.371682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.371754 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.372052 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.372053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.372054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.372208 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.372273 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.372838 (+ 565us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.372920 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:10:01.373543 16993 catalog_manager.cc:1347] Generated new cluster ID: 5e18bb8ca6be4013873a113990bec8b5
I20260929 19:10:01.373600 16993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:01.382920 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.363220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58516 (local address 127.26.78.254:44739)
0929 19:10:01.363262 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.363264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.371546 (+ 8282us) server_negotiation.cc:407] Connection header received
0929 19:10:01.371703 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.371704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.371714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.371746 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.374243 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.374987 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.376545 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.376847 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.376890 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.379303 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.379305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.379306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.379313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.381117 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.381706 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.381707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.381707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.381774 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.382180 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.382181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.382181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.382334 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.382430 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.382740 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.382820 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 29 19:10:01 dist-test-slave-xz51 krb5kdc[16727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709001, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:10:01.393537 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.373164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:37239 (local address 127.0.0.1:36174)
0929 19:10:01.373239 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:01.373248 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:01.373284 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:01.385875 (+ 12591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:01.385877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:01.386059 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:01.386112 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.386115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.387061 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.387062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.387796 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:01.387798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.388286 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.388287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:01.388343 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.388778 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:01.388783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:01.390299 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:01.392321 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.392322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.392323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.392550 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.392789 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:01.392790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:01.392790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:01.392862 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:01.393196 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:01.393197 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:01.393264 (+ 67us) client_negotiation.cc:769] Sending connection context
0929 19:10:01.393331 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 19:10:01.393430 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 19:10:01.394296 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:01.373347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36174 (local address 127.26.78.252:37239)
0929 19:10:01.373379 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.373381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.373387 (+ 6us) server_negotiation.cc:407] Connection header received
0929 19:10:01.373405 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.373405 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.373413 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.373459 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.386235 (+ 12776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.386974 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.387884 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:01.388204 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:01.388287 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:01.390381 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:01.390382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:01.390382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:01.390390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:01.392198 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.392642 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.392643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.392643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.392712 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:01.392936 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:01.392936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:01.392937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:01.393085 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:01.393155 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:01.394107 (+ 952us) server_negotiation.cc:299] Negotiation successful
0929 19:10:01.394187 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:10:01.396010 16883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad: Bootstrap starting.
I20260929 19:10:01.396775 16883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:01.397091 16883 log.cc:824] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:01.407101 16883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad: No bootstrap required, opened a new log
I20260929 19:10:01.409088 16883 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
I20260929 19:10:01.409256 16883 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5cfb79f954141f6ad2e78bac19f87ad, State: Initialized, Role: FOLLOWER
I20260929 19:10:01.409345 16883 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } }
I20260929 19:10:01.409723 16883 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:01.409983 16857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "625f19e6ff7c4659b9189d17555fe7a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c5cfb79f954141f6ad2e78bac19f87ad"
I20260929 19:10:01.410363 16857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:01.411321 16857 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625f19e6ff7c4659b9189d17555fe7a5 in term 1.
I20260929 19:10:01.411810 16856 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [term 1 FOLLOWER]: Refusing update from remote peer 625f19e6ff7c4659b9189d17555fe7a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:01.412408 16991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:01.412990 16993 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:01.413764 16993 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:01.415086 17021 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625f19e6ff7c4659b9189d17555fe7a5. Latest consensus state: current_term: 1 leader_uuid: "625f19e6ff7c4659b9189d17555fe7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.415282 17021 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:01.415393 17021 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "625f19e6ff7c4659b9189d17555fe7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.415490 17021 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:01.416025 17019 mvcc.cc:205] Tried to move back new op lower bound from 7334744069774221312 to 7334744069550407680. Current Snapshot: MvccSnapshot[applied={T|T < 7334744069589417984}]
I20260929 19:10:01.414288 17007 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.416940 17007 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:01.418416 17022 catalog_manager.cc:1259] Loaded cluster ID: 5e18bb8ca6be4013873a113990bec8b5
I20260929 19:10:01.418526 17022 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad: loading cluster ID for follower catalog manager: success
I20260929 19:10:01.419721 17022 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad: acquiring CA information for follower catalog manager: success
W20260929 19:10:01.419926 17022 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260929 19:10:01.432600 16991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "625f19e6ff7c4659b9189d17555fe7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f19e6ff7c4659b9189d17555fe7a5" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44739 } } peers { permanent_uuid: "c5cfb79f954141f6ad2e78bac19f87ad" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 36345 } } peers { permanent_uuid: "a865839a9e62463e92ec17ad7fea26c2" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 37239 } } }
I20260929 19:10:01.432730 16991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:01.435702 16993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 625f19e6ff7c4659b9189d17555fe7a5: Generated new TSK 0
I20260929 19:10:01.435913 16993 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:10:01.466089 16811 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.464309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58526 (local address 127.26.78.254:44739)
0929 19:10:01.464359 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.464362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.465320 (+ 958us) server_negotiation.cc:407] Connection header received
0929 19:10:01.465461 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.465462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.465473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.465505 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.466017 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58526: BlockingRecv error: recv got EOF from 127.0.0.1:58526 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260929 19:10:01.468102 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.464597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.26.78.253:36345)
0929 19:10:01.464634 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.464635 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.466334 (+ 1699us) server_negotiation.cc:407] Connection header received
0929 19:10:01.466447 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.466513 (+ 66us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.466526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.466559 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.467998 (+ 1439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38884: BlockingRecv error: recv got EOF from 127.0.0.1:38884 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 19:10:01.468879 16962 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.465054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36186 (local address 127.26.78.252:37239)
0929 19:10:01.465093 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.465095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.468340 (+ 3245us) server_negotiation.cc:407] Connection header received
0929 19:10:01.468460 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.468461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.468472 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.468503 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.468821 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36186: BlockingRecv error: recv got EOF from 127.0.0.1:36186 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 19:10:01.471545 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.470617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38890 (local address 127.26.78.253:36345)
0929 19:10:01.470661 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.470663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.470935 (+ 272us) server_negotiation.cc:407] Connection header received
0929 19:10:01.471062 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.471063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.471081 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.471114 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.471500 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38890: BlockingRecv error: recv got EOF from 127.0.0.1:38890 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 19:10:01.474758 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.473895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38898 (local address 127.26.78.253:36345)
0929 19:10:01.473932 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.473934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.474202 (+ 268us) server_negotiation.cc:407] Connection header received
0929 19:10:01.474324 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.474325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.474336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.474367 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.474708 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38898: BlockingRecv error: recv got EOF from 127.0.0.1:38898 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 19:10:01.479007 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.478122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.26.78.253:36345)
0929 19:10:01.478161 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.478163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.478447 (+ 284us) server_negotiation.cc:407] Connection header received
0929 19:10:01.478557 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.478558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.478569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.478601 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.478953 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38906: BlockingRecv error: recv got EOF from 127.0.0.1:38906 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 19:10:01.485210 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.484190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.26.78.253:36345)
0929 19:10:01.484232 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.484234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.484603 (+ 369us) server_negotiation.cc:407] Connection header received
0929 19:10:01.484720 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.484721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.484734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.484769 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.485153 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38916: BlockingRecv error: recv got EOF from 127.0.0.1:38916 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 19:10:01.495754 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.494721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38918 (local address 127.26.78.253:36345)
0929 19:10:01.494769 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.494771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.495086 (+ 315us) server_negotiation.cc:407] Connection header received
0929 19:10:01.495206 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.495207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.495220 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.495271 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.495690 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38918: BlockingRecv error: recv got EOF from 127.0.0.1:38918 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 19:10:01.514533 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.513344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.26.78.253:36345)
0929 19:10:01.513391 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.513393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.513744 (+ 351us) server_negotiation.cc:407] Connection header received
0929 19:10:01.513938 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.513940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.513954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.514014 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.514466 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38922: BlockingRecv error: recv got EOF from 127.0.0.1:38922 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 19:10:01.548864 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.548052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.26.78.253:36345)
0929 19:10:01.548103 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.548106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.548259 (+ 153us) server_negotiation.cc:407] Connection header received
0929 19:10:01.548302 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.548303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.548315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.548399 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.548788 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38930: BlockingRecv error: recv got EOF from 127.0.0.1:38930 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260929 19:10:01.614959 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.614137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38932 (local address 127.26.78.253:36345)
0929 19:10:01.614228 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.614231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.614344 (+ 113us) server_negotiation.cc:407] Connection header received
0929 19:10:01.614434 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.614435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.614447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.614506 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.614891 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38932: BlockingRecv error: recv got EOF from 127.0.0.1:38932 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 19:10:01.744923 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:01.744123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.26.78.253:36345)
0929 19:10:01.744217 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:01.744219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:01.744320 (+ 101us) server_negotiation.cc:407] Connection header received
0929 19:10:01.744425 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:01.744426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:01.744438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:01.744510 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:01.744854 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38942: BlockingRecv error: recv got EOF from 127.0.0.1:38942 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260929 19:10:02.002804 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:02.002151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38954 (local address 127.26.78.253:36345)
0929 19:10:02.002215 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:02.002217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:02.002233 (+ 16us) server_negotiation.cc:407] Connection header received
0929 19:10:02.002260 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:02.002261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:02.002272 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:02.002334 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:02.002719 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38954: BlockingRecv error: recv got EOF from 127.0.0.1:38954 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:10:02.325291 16975 catalog_manager.cc:1259] Loaded cluster ID: 5e18bb8ca6be4013873a113990bec8b5
I20260929 19:10:02.325371 16975 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: loading cluster ID for follower catalog manager: success
I20260929 19:10:02.326213 16975 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: acquiring CA information for follower catalog manager: success
I20260929 19:10:02.326535 16975 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a865839a9e62463e92ec17ad7fea26c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:10:02.420565 17022 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c5cfb79f954141f6ad2e78bac19f87ad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:10:02.518827 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:02.516029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.26.78.253:36345)
0929 19:10:02.516141 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:02.516144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:02.516222 (+ 78us) server_negotiation.cc:407] Connection header received
0929 19:10:02.516318 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:02.516319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:02.516379 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:02.516475 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:02.516626 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.517186 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.518005 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.518227 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.518290 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:02.518469 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:02.518587 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:02.518674 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:02.518697 (+ 23us) server_negotiation.cc:299] Negotiation successful
0929 19:10:02.518719 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260929 19:10:02.523057 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:02.520663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36188 (local address 127.26.78.252:37239)
0929 19:10:02.520766 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:02.520768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:02.520778 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:02.520814 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:02.520815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:02.520850 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:02.520925 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:02.521106 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.521585 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.522318 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.522509 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.522570 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:02.522705 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:02.522807 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:02.522868 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:02.522925 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 19:10:02.522943 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":58,"threads_started":1}
I20260929 19:10:02.527562 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:02.524749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.26.78.253:36345)
0929 19:10:02.524787 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:02.524789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:02.524981 (+ 192us) server_negotiation.cc:407] Connection header received
0929 19:10:02.525096 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:02.525097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:02.525108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:02.525162 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:02.525281 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.526059 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.526899 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.527120 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.527177 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:02.527290 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:02.527320 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:02.527367 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:02.527465 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 19:10:02.527490 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:10:02.527851 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:02.524927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.26.78.252:37239)
0929 19:10:02.525046 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:02.525048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:02.525161 (+ 113us) server_negotiation.cc:407] Connection header received
0929 19:10:02.525187 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:02.525188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:02.525198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:02.525240 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:02.525393 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.526103 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.527117 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.527424 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.527493 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:02.527594 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:02.527623 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:02.527669 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:02.527764 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 19:10:02.527789 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260929 19:10:02.528249 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:02.524917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58528 (local address 127.26.78.254:44739)
0929 19:10:02.525015 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:02.525018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:02.525027 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:02.525075 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:02.525076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:02.525123 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:02.525234 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:02.525342 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.525917 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.526951 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:02.527264 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:02.527330 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:02.527478 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:02.527586 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:02.527643 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:02.528149 (+ 506us) server_negotiation.cc:299] Negotiation successful
0929 19:10:02.528175 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":55,"threads_started":1}
I20260929 19:10:02.529727 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16740
I20260929 19:10:02.535140 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16815
I20260929 19:10:02.540071 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 16890
2026-09-29T19:10:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2824 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:02 dist-test-slave-xz51 krb5kdc[17108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:02 dist-test-slave-xz51 krb5kdc[17108](info): set up 2 sockets
Sep 29 19:10:02 dist-test-slave-xz51 krb5kdc[17108](info): commencing operation
krb5kdc: starting...
W20260929 19:10:03.574214 26939 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/test-user.keytab.
Sep 29 19:10:03 dist-test-slave-xz51 krb5kdc[17108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.78.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
Entry for principal kudu/127.26.78.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab.
I20260929 19:10:03.691018 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44831
--webserver_interface=127.26.78.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44831,127.26.78.253:41749,127.26.78.252:34305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:03.774952 17121 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:10:03.775195 17121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:03.775225 17121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:03.775265 17121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:03.776381 17121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:03.776445 17121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:03.776469 17121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:03.776512 17121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:03.776540 17121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:03.778065 17121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44831,127.26.78.253:41749,127.26.78.252:34305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.254.keytab
--principal=kudu/127.26.78.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.254:44831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.78.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:03.778345 17121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:03.778571 17121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:03.783469 17129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:03.783432 17126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:03.783474 17127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:03.783593 17121 server_base.cc:1034] running on GCE node
I20260929 19:10:03.783951 17121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:03.784242 17121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:03.785389 17121 hybrid_clock.cc:648] HybridClock initialized: now 1790709003785370 us; error 44 us; skew 500 ppm
Sep 29 19:10:03 dist-test-slave-xz51 krb5kdc[17108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:03.787830 17121 init.cc:377] Logged in from keytab as kudu/127.26.78.254@KRBTEST.COM (short username kudu)
I20260929 19:10:03.788200 17121 webserver.cc:460] Webserver started at http://127.26.78.254:46735/ using document root <none> and password file <none>
I20260929 19:10:03.788375 17121 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:03.788419 17121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:03.788506 17121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:03.789494 17121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data/instance:
uuid: "0c97f3082b23427f92de3213550dcbf3"
format_stamp: "Formatted at 2026-09-29 19:10:03 on dist-test-slave-xz51"
server_key: "338bfd96cf73aab0b665e9dab6424e9a"
server_key_iv: "574decadb7fbdc135d0c81c927e10206"
server_key_version: "encryptionkey@0"
I20260929 19:10:03.789783 17121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/wal/instance:
uuid: "0c97f3082b23427f92de3213550dcbf3"
format_stamp: "Formatted at 2026-09-29 19:10:03 on dist-test-slave-xz51"
server_key: "338bfd96cf73aab0b665e9dab6424e9a"
server_key_iv: "574decadb7fbdc135d0c81c927e10206"
server_key_version: "encryptionkey@0"
I20260929 19:10:03.791087 17121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:03.791769 17136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:03.791919 17121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:03.792008 17121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/wal
uuid: "0c97f3082b23427f92de3213550dcbf3"
format_stamp: "Formatted at 2026-09-29 19:10:03 on dist-test-slave-xz51"
server_key: "338bfd96cf73aab0b665e9dab6424e9a"
server_key_iv: "574decadb7fbdc135d0c81c927e10206"
server_key_version: "encryptionkey@0"
I20260929 19:10:03.792064 17121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:03.799453 17121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:03.801334 17121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:03.801458 17121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:03.805150 17121 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.254:44831
I20260929 19:10:03.805195 17188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.254:44831 every 8 connection(s)
I20260929 19:10:03.805548 17121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:03.805758 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 17121
I20260929 19:10:03.805955 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-0/wal/instance
I20260929 19:10:03.806210 26939 external_mini_cluster.cc:1428] Setting key 19a1d7bce559809a9c4fc3f09c6864b0
I20260929 19:10:03.806828 17189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:03.808682 17189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.78.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.253@KRBTEST.COM" created.
Sep 29 19:10:03 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:10:03.820477 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.809453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.26.78.254:44831)
0929 19:10:03.809608 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:03.809617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:03.809735 (+ 118us) server_negotiation.cc:407] Connection header received
0929 19:10:03.810349 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:03.810351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:03.810367 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:03.810461 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:03.811004 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.811817 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.812703 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.813009 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.813121 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:03.816983 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:03.816990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:03.816991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:03.817002 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:03.818904 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:03.819282 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:03.819285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:03.819285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:03.819364 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:03.819604 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:03.819605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:03.819605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:03.819828 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:03.819917 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:03.820100 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 19:10:03.820231 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260929 19:10:03.820479 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.809406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44831 (local address 127.0.0.1:41052)
0929 19:10:03.809596 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.809636 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:03.810190 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:03.810473 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:03.810475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:03.810752 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:03.810898 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:03.810904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.811910 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.811911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:03.812603 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:03.812605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.813107 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.813108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:03.813170 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:03.813684 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:03.813693 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:03.816880 (+ 3187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:03.818994 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:03.819000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:03.819003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:03.819190 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:03.819433 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:03.819433 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:03.819434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:03.819538 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:03.819901 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:03.819906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:03.819968 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 19:10:03.820068 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 19:10:03.820295 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":84,"thread_start_us":96,"threads_started":1}
I20260929 19:10:03.821162 17189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:03.821498 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.821388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53136)
0929 19:10:03.821430 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.821472 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 19:10:03.821623 17137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
I20260929 19:10:03.822036 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.821947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53152)
0929 19:10:03.821990 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.822013 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 19:10:03.822137 17189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:41749: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab.
Entry for principal kudu/127.26.78.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab.
I20260929 19:10:03.830742 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:41749
--webserver_interface=127.26.78.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab
--principal=kudu/127.26.78.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44831,127.26.78.253:41749,127.26.78.252:34305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:03.871345 17189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } attempt: 1
I20260929 19:10:03.872021 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.871879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53164)
0929 19:10:03.871949 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.871993 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 19:10:03.872365 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.872290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53166)
0929 19:10:03.872327 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.872349 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 19:10:03.872473 17189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:41749: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
W20260929 19:10:03.911676 17196 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:10:03.911902 17196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:03.911921 17196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:03.911972 17196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:03.913094 17196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:03.913133 17196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:03.913148 17196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:03.913195 17196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:03.913214 17196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:03.914737 17196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44831,127.26.78.253:41749,127.26.78.252:34305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.253.keytab
--principal=kudu/127.26.78.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.253:41749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.78.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:03.915010 17196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:03.915234 17196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:10:03.920100 17196 server_base.cc:1034] running on GCE node
W20260929 19:10:03.920078 17202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:03.920205 17201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:03.920092 17204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:03.920502 17196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:03.920826 17196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:03.921986 17196 hybrid_clock.cc:648] HybridClock initialized: now 1790709003921970 us; error 30 us; skew 500 ppm
Sep 29 19:10:03 dist-test-slave-xz51 krb5kdc[17108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:03.924475 17196 init.cc:377] Logged in from keytab as kudu/127.26.78.253@KRBTEST.COM (short username kudu)
I20260929 19:10:03.924942 17196 webserver.cc:460] Webserver started at http://127.26.78.253:36009/ using document root <none> and password file <none>
I20260929 19:10:03.925166 17196 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:03.925231 17196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:03.925374 17196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:03.926504 17196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data/instance:
uuid: "773fe074d21e4601b5dd06257ed39b08"
format_stamp: "Formatted at 2026-09-29 19:10:03 on dist-test-slave-xz51"
server_key: "2165357fb0d698d43a16fc3d34d87675"
server_key_iv: "144bc261c89aeb4b0b680242cbfe4834"
server_key_version: "encryptionkey@0"
I20260929 19:10:03.926859 17196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/wal/instance:
uuid: "773fe074d21e4601b5dd06257ed39b08"
format_stamp: "Formatted at 2026-09-29 19:10:03 on dist-test-slave-xz51"
server_key: "2165357fb0d698d43a16fc3d34d87675"
server_key_iv: "144bc261c89aeb4b0b680242cbfe4834"
server_key_version: "encryptionkey@0"
I20260929 19:10:03.928210 17196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:03.928993 17211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:03.929176 17196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:03.929275 17196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/wal
uuid: "773fe074d21e4601b5dd06257ed39b08"
format_stamp: "Formatted at 2026-09-29 19:10:03 on dist-test-slave-xz51"
server_key: "2165357fb0d698d43a16fc3d34d87675"
server_key_iv: "144bc261c89aeb4b0b680242cbfe4834"
server_key_version: "encryptionkey@0"
I20260929 19:10:03.929347 17196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:03.940614 17189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } attempt: 2
I20260929 19:10:03.941056 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.940927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53172)
0929 19:10:03.940990 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.941026 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 19:10:03.941381 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.941289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53174)
0929 19:10:03.941332 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.941356 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 19:10:03.941478 17189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.253:41749: Network error: Client connection negotiation failed: client connection to 127.26.78.253:41749: connect: Connection refused (error 111)
I20260929 19:10:03.945753 17196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:03.948062 17196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:03.948225 17196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:03.952128 17196 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.253:41749
I20260929 19:10:03.952172 17263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.253:41749 every 8 connection(s)
I20260929 19:10:03.952555 17196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/data/info.pb
I20260929 19:10:03.953521 17264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:03.955045 17264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:03.955436 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 17196
I20260929 19:10:03.955546 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-1/wal/instance
I20260929 19:10:03.955772 26939 external_mini_cluster.cc:1428] Setting key 0b4f1f559afcb2fe103cd6171ef25c5f
Sep 29 19:10:03 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.78.252@KRBTEST.COM; defaulting to no policy
I20260929 19:10:03.964289 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.955803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.26.78.254:44831)
0929 19:10:03.955853 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:03.955855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:03.956143 (+ 288us) server_negotiation.cc:407] Connection header received
0929 19:10:03.956650 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:03.956652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:03.956668 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:03.956731 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:03.957359 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.957905 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.958909 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.959168 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.959231 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:03.961407 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:03.961410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:03.961410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:03.961418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:03.962840 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:03.963281 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:03.963282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:03.963282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:03.963341 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:03.963602 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:03.963603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:03.963603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:03.963723 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:03.963796 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:03.964109 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 19:10:03.964195 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:10:03.964488 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.955805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44831 (local address 127.0.0.1:41058)
0929 19:10:03.956022 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.956059 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:03.956542 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:03.956805 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:03.956808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:03.957068 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:03.957266 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:03.957270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.958017 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.958018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:03.958806 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:03.958809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.959254 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.959255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:03.959384 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:03.959950 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:03.959958 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:03.961310 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:03.962963 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:03.962971 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:03.962974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:03.963205 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:03.963414 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:03.963415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:03.963415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:03.963530 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:03.963788 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:03.963796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:03.963972 (+ 176us) client_negotiation.cc:769] Sending connection context
0929 19:10:03.964077 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 19:10:03.964238 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":132,"threads_started":1}
I20260929 19:10:03.965035 17264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.78.252@KRBTEST.COM" created.
Sep 29 19:10:03 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:10:03.972738 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.965286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53178)
0929 19:10:03.965334 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.965345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:03.965394 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:03.965674 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:03.965675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:03.965816 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:03.965909 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:03.965910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.966755 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.966756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:03.967587 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:03.967589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.968066 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.968068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:03.968153 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:03.968549 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:03.968554 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:03.969959 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:03.971698 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:03.971699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:03.971699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:03.971859 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:03.972057 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:03.972057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:03.972058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:03.972110 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:03.972407 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:03.972407 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:03.972447 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 19:10:03.972527 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 19:10:03.972625 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260929 19:10:03.972803 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.965365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53178 (local address 127.26.78.253:41749)
0929 19:10:03.965543 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:03.965548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:03.965569 (+ 21us) server_negotiation.cc:407] Connection header received
0929 19:10:03.965586 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:03.965587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:03.965598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:03.965675 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:03.965991 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.966656 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.967678 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:03.967989 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:03.968049 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:03.970048 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:03.970056 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:03.970056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:03.970062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:03.971599 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:03.971938 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:03.971941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:03.971942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:03.971994 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:03.972176 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:03.972177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:03.972177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:03.972320 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:03.972409 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:03.972557 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 19:10:03.972678 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260929 19:10:03.973384 17264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:03.973747 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.973641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34378)
0929 19:10:03.973674 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.973726 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260929 19:10:03.973891 17213 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
I20260929 19:10:03.974320 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:03.974231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34394)
0929 19:10:03.974281 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:03.974304 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 19:10:03.974421 17264 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:34305: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.78.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab.
Entry for principal kudu/127.26.78.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab.
I20260929 19:10:03.980820 26939 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZMrn7N/build/release/bin/kudu
/tmp/dist-test-taskZMrn7N/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:34305
--webserver_interface=127.26.78.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab
--principal=kudu/127.26.78.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.78.212:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.78.254:44831,127.26.78.253:41749,127.26.78.252:34305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:04.023658 17264 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } attempt: 1
I20260929 19:10:04.024186 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.024032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34400)
0929 19:10:04.024100 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.024150 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 19:10:04.024601 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.024504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34412)
0929 19:10:04.024550 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.024577 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 19:10:04.024718 17264 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:34305: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
I20260929 19:10:04.032619 17189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } attempt: 3
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:10:04.039578 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.032938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53188)
0929 19:10:04.032995 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.033010 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:04.033075 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:04.033250 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:04.033251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:04.033416 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:04.033480 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.033481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.034179 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.034181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.034894 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.034904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.035260 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.035262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.035335 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.035838 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:04.035843 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:04.037026 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:04.038578 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.038579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.038580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.038748 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.038953 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.038954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.038954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.039011 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.039250 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:04.039252 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:04.039296 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 19:10:04.039377 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:10:04.039475 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 19:10:04.039578 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.033011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.26.78.253:41749)
0929 19:10:04.033068 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.033070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.033080 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:04.033161 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.033162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.033171 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.033241 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.033558 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.034100 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.034991 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.035191 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.035264 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.037112 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.037113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.037114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.037121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.038498 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.038831 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.038833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.038833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.038886 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.039077 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.039078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.039078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.039187 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.039258 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.039419 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.039492 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:10:04.040230 17189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:04.040581 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.040477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34414)
0929 19:10:04.040522 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.040548 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:10:04.040923 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.040842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34418)
0929 19:10:04.040878 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.040901 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260929 19:10:04.041034 17189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:34305: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
W20260929 19:10:04.063580 17271 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:10:04.063800 17271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:04.063818 17271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:04.063838 17271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:04.065052 17271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:04.065094 17271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:04.065109 17271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:04.065121 17271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:04.065163 17271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:04.066725 17271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.78.212:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.78.254:44831,127.26.78.253:41749,127.26.78.252:34305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/krb5kdc/kudu_127.26.78.252.keytab
--principal=kudu/127.26.78.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.78.252:34305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.78.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:04.066991 17271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:04.067231 17271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:10:04.072186 17189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } attempt: 1
W20260929 19:10:04.072237 17279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:04.072431 17277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:04.072265 17276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:04.072316 17271 server_base.cc:1034] running on GCE node
I20260929 19:10:04.072594 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.072480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34434)
0929 19:10:04.072533 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.072562 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 19:10:04.072779 17271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:04.072992 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.072852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34444)
0929 19:10:04.072924 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.072961 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 19:10:04.073096 17189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:34305: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
I20260929 19:10:04.073107 17271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:04.074257 17271 hybrid_clock.cc:648] HybridClock initialized: now 1790709004074236 us; error 35 us; skew 500 ppm
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709004, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:04.076749 17271 init.cc:377] Logged in from keytab as kudu/127.26.78.252@KRBTEST.COM (short username kudu)
I20260929 19:10:04.077234 17271 webserver.cc:460] Webserver started at http://127.26.78.252:40743/ using document root <none> and password file <none>
I20260929 19:10:04.077476 17271 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:04.077541 17271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:04.077687 17271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:04.078804 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data/instance:
uuid: "07308e1c21644539b6b0e5cc63313bb0"
format_stamp: "Formatted at 2026-09-29 19:10:04 on dist-test-slave-xz51"
server_key: "d4b1dcf265df0502c8905a58ec217b98"
server_key_iv: "548f92699bd3c04578ebf34161635ec8"
server_key_version: "encryptionkey@0"
I20260929 19:10:04.079162 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/wal/instance:
uuid: "07308e1c21644539b6b0e5cc63313bb0"
format_stamp: "Formatted at 2026-09-29 19:10:04 on dist-test-slave-xz51"
server_key: "d4b1dcf265df0502c8905a58ec217b98"
server_key_iv: "548f92699bd3c04578ebf34161635ec8"
server_key_version: "encryptionkey@0"
I20260929 19:10:04.080469 17271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:04.081177 17286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:04.081341 17271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:04.081450 17271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data,/tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/wal
uuid: "07308e1c21644539b6b0e5cc63313bb0"
format_stamp: "Formatted at 2026-09-29 19:10:04 on dist-test-slave-xz51"
server_key: "d4b1dcf265df0502c8905a58ec217b98"
server_key_iv: "548f92699bd3c04578ebf34161635ec8"
server_key_version: "encryptionkey@0"
I20260929 19:10:04.081530 17271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:04.092866 17264 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } attempt: 2
I20260929 19:10:04.093286 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.093165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34460)
0929 19:10:04.093223 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.093258 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 19:10:04.093638 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.093544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34472)
0929 19:10:04.093586 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.093622 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 19:10:04.093712 17264 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.78.252:34305: Network error: Client connection negotiation failed: client connection to 127.26.78.252:34305: connect: Connection refused (error 111)
I20260929 19:10:04.107583 17271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:04.110008 17271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:04.110175 17271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:04.114145 17271 rpc_server.cc:307] RPC server started. Bound to: 127.26.78.252:34305
I20260929 19:10:04.114197 17338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.78.252:34305 every 8 connection(s)
I20260929 19:10:04.114573 17271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/data/info.pb
I20260929 19:10:04.115595 17339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:04.115685 26939 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZMrn7N/build/release/bin/kudu as pid 17271
I20260929 19:10:04.115790 26939 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZMrn7N/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790708710206825-26939-0/minicluster-data/master-2/wal/instance
I20260929 19:10:04.116092 26939 external_mini_cluster.cc:1428] Setting key fe9bf6d84ff52f28e2ba7072c60b51b2
I20260929 19:10:04.118230 17339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709004, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:10:04.126639 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.118940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.26.78.254:44831)
0929 19:10:04.119001 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.119004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.119156 (+ 152us) server_negotiation.cc:407] Connection header received
0929 19:10:04.119215 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.119216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.119227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.119279 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.119725 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.120183 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.121037 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.121209 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.121288 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.123816 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.123818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.123818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.123826 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.125337 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.125731 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.125732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.125732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.125785 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.126073 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.126074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.126074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.126196 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.126266 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.126463 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.126546 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:10:04.126935 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.118866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.254:44831 (local address 127.0.0.1:41062)
0929 19:10:04.119073 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.119101 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:04.119154 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:04.119309 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:04.119311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:04.119564 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:04.119648 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.119651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.120269 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.120270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.120954 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.120956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.121314 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.121315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.121385 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.121812 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:04.121860 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:04.123724 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:04.125406 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.125412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.125414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.125664 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.125878 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.125878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.125879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.126000 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.126261 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:04.126265 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:04.126319 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 19:10:04.126428 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 19:10:04.126610 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":99,"threads_started":1}
I20260929 19:10:04.126937 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.116739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34484 (local address 127.26.78.252:34305)
0929 19:10:04.116917 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.116927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.116960 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:10:04.117459 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.117468 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.117533 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.117687 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.118317 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.119561 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.120424 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.120813 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.120908 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.123290 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.123298 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.123298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.123305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.125252 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.125665 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.125667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.125667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.125783 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.126010 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.126011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.126011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.126301 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.126440 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.126539 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.126682 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260929 19:10:04.128085 17339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709004, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.254@KRBTEST.COM
I20260929 19:10:04.134639 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.128415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53200 (local address 127.26.78.253:41749)
0929 19:10:04.128476 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.128478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.128486 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:04.128509 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.128509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.128518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.128578 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.128844 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.129324 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.130003 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.130201 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.130268 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.132102 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.132103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.132103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.132109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.133533 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.133895 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.133896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.133896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.133955 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.134118 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.134119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.134120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.134256 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.134325 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.134466 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.134551 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:10:04.134640 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.128338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.253:41749 (local address 127.0.0.1:53200)
0929 19:10:04.128381 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.128389 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:04.128419 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:04.128592 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:04.128593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:04.128737 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:04.128769 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.128770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.129400 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.129401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.129927 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.129929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.130273 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.130273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.130320 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.130818 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:04.130823 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:04.132022 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:04.133592 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.133593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.133593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.133803 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.134022 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.134023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.134023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.134069 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.134330 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:04.134330 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:04.134363 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 19:10:04.134436 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 19:10:04.134504 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 19:10:04.135053 17339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:04.135931 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.127985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.26.78.254:44831)
0929 19:10:04.128032 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.128035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.128044 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:04.128087 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.128088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.128098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.128160 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.128669 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.129192 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.129924 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.130158 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.130205 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.132775 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.132776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.132777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.132784 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.134476 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.134947 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.134948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.134948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.135005 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.135380 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.135381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.135382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.135529 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.135659 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.135753 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.135832 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709004, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.252@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.78.253@KRBTEST.COM
I20260929 19:10:04.141680 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.135299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34488)
0929 19:10:04.135345 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.135355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:04.135389 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:04.135505 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:04.135506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:04.135643 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:04.135675 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.135677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.136487 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.136488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.137046 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.137064 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.137405 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.137406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.137459 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.137805 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:04.137814 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:04.139153 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:04.140599 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.140600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.140600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.140807 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.141006 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.141006 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.141007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.141058 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.141333 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:04.141334 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:04.141391 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 19:10:04.141493 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 19:10:04.141597 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 19:10:04.141793 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.135325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.26.78.252:34305)
0929 19:10:04.135372 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.135374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.135382 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:04.135433 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.135433 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.135442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.135503 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.135741 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.136419 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.137144 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.137341 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.137391 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.139234 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.139235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.139236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.139243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.140521 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.140876 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.140877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.140878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.140935 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.141117 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.141118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.141119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.141253 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.141311 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.141543 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.141660 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":27}
I20260929 19:10:04.143301 17339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: Bootstrap starting.
I20260929 19:10:04.144339 17339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:04.144610 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.136533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.26.78.253:41749)
0929 19:10:04.136569 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.136570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.136577 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:04.136626 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.136627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.136635 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.136687 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.137090 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.137661 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.138411 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.138588 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.138641 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.141245 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.141246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.141246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.141252 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.143299 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.143769 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.143770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.143771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.143852 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.144075 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.144076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.144077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.144213 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.144392 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.144428 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.144512 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:10:04.144675 17339 log.cc:824] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:04.145632 17339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: No bootstrap required, opened a new log
I20260929 19:10:04.146091 26939 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:10:04.147315 17339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.147465 17339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07308e1c21644539b6b0e5cc63313bb0, State: Initialized, Role: FOLLOWER
I20260929 19:10:04.147557 17339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.146909 17163 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0c97f3082b23427f92de3213550dcbf3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41074
I20260929 19:10:04.147806 17345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.147884 17339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:04.147891 17345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:04.148252 17189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } attempt: 2
W20260929 19:10:04.149724 17356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:04.149783 17356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.254@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:10:04.149991 17271 master_runner.cc:186] Error getting master registration for 127.26.78.254:44831: OK, instance_id { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" instance_seqno: 1790709003804159 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:10:04.153012 17271 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:10:04.155982 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.148503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34494)
0929 19:10:04.148538 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.148551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:04.148591 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:04.148749 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:04.148750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:04.148922 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:04.148981 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.148984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.149772 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.149773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.150574 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.150577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.151011 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.151013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.151082 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.151555 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:04.151561 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:04.153145 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:04.154861 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.154862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.154863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.155101 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.155308 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.155308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.155309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.155368 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.155674 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:04.155675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:04.155721 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 19:10:04.155785 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:10:04.155873 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 19:10:04.155989 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.148557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34494 (local address 127.26.78.252:34305)
0929 19:10:04.148612 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.148614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.148626 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:04.148662 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.148663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.148672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.148735 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.149056 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.149691 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.150665 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.150933 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.151012 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.153224 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.153226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.153226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.153233 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.154778 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.155176 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.155177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.155178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.155237 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.155431 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.155432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.155432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.155601 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.155668 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.155823 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.155903 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:10:04.158308 17189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3: Bootstrap starting.
I20260929 19:10:04.159107 17189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:04.159428 17189 log.cc:824] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:04.160339 17189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3: No bootstrap required, opened a new log
I20260929 19:10:04.161916 17189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.162093 17189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c97f3082b23427f92de3213550dcbf3, State: Initialized, Role: FOLLOWER
I20260929 19:10:04.162189 17189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.162532 17189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:04.162505 17359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.162551 17163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 19:10:04.162604 17359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:04.162638 17163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:04.163401 17163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.163903 17163 leader_election.cc:288] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [CANDIDATE]: Term 1 election: Requested vote from peers 773fe074d21e4601b5dd06257ed39b08 (127.26.78.253:41749), 07308e1c21644539b6b0e5cc63313bb0 (127.26.78.252:34305)
I20260929 19:10:04.164090 17313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c97f3082b23427f92de3213550dcbf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "07308e1c21644539b6b0e5cc63313bb0"
I20260929 19:10:04.164227 17313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:04.165186 17313 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c97f3082b23427f92de3213550dcbf3 in term 1.
I20260929 19:10:04.165724 17138 leader_election.cc:302] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07308e1c21644539b6b0e5cc63313bb0, 0c97f3082b23427f92de3213550dcbf3; no voters:
I20260929 19:10:04.165895 17359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:04.166049 17359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [term 1 LEADER]: Becoming Leader. State: Replica: 0c97f3082b23427f92de3213550dcbf3, State: Running, Role: LEADER
I20260929 19:10:04.166133 17359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.166543 17359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c97f3082b23427f92de3213550dcbf3. Latest consensus state: current_term: 1 leader_uuid: "0c97f3082b23427f92de3213550dcbf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.166625 17359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:04.166916 17380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:04.167835 17380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:04.167022 17121 master_runner.cc:186] Error getting master registration for 127.26.78.253:41749: OK, instance_id { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" instance_seqno: 1790709003951038 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:10:04.169344 17121 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:10:04.174772 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.166212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.26.78.254:44831)
0929 19:10:04.166251 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.166253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.166593 (+ 340us) server_negotiation.cc:407] Connection header received
0929 19:10:04.166678 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.166679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.166690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.166719 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.167540 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.168242 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.169254 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.169536 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.169580 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.171506 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.171507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.171507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.171514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.173462 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.173890 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.173891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.173891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.173960 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.174175 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.174176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.174176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.174308 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.174382 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.174609 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.174687 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:10:04.179471 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.166830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.26.78.253:41749)
0929 19:10:04.166875 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.166877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.166887 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:04.166911 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.166912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.166923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.166988 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.170733 (+ 3745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.171468 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.172487 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.172792 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.172831 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.176304 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.176306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.176306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.176313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.178216 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.178653 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.178654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.178655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.178709 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.178894 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.178895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.178896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.179016 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.179091 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.179271 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.179366 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:10:04.184489 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.165888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34498 (local address 127.26.78.252:34305)
0929 19:10:04.169883 (+ 3995us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.169885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.169894 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:04.169914 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.169915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.169925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.169985 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.174593 (+ 4608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.175309 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.176613 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.176925 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.176968 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.181692 (+ 4724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.181693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.181693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.181700 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.183242 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.183639 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.183640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.183641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.183693 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.183876 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.183876 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.183877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.183999 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.184112 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.184310 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.184385 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3920}
I20260929 19:10:04.184829 17264 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } attempt: 3
I20260929 19:10:04.185626 17313 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [term 1 FOLLOWER]: Refusing update from remote peer 0c97f3082b23427f92de3213550dcbf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:04.185954 17359 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:04.187204 17345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c97f3082b23427f92de3213550dcbf3. Latest consensus state: current_term: 1 leader_uuid: "0c97f3082b23427f92de3213550dcbf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.187287 17345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:04.188540 17359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c97f3082b23427f92de3213550dcbf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.188624 17359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:04.188822 17382 mvcc.cc:205] Tried to move back new op lower bound from 7334744081141788672 to 7334744081065144320. Current Snapshot: MvccSnapshot[applied={T|T < 7334744081141788672}]
I20260929 19:10:04.189738 17345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c97f3082b23427f92de3213550dcbf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.189814 17345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0 [sys.catalog]: This master's current role is: FOLLOWER
Sep 29 19:10:04 dist-test-slave-xz51 krb5kdc[17108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709003, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.78.253@KRBTEST.COM for kudu/127.26.78.252@KRBTEST.COM
I20260929 19:10:04.191035 17383 mvcc.cc:205] Tried to move back new op lower bound from 7334744081141788672 to 7334744081065144320. Current Snapshot: MvccSnapshot[applied={T|T < 7334744081141788672}]
I20260929 19:10:04.191845 17380 catalog_manager.cc:1347] Generated new cluster ID: b274b90d5ea7478fa66e1f74551bb8a1
I20260929 19:10:04.191896 17380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:04.193751 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.185096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.78.252:34305 (local address 127.0.0.1:34514)
0929 19:10:04.185176 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:04.185186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:04.185223 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:04.185536 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:04.185537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:04.185698 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:04.185747 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.185749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.186939 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.186940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.187715 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:04.187718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.188448 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.188449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:04.188508 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.189053 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:04.189059 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:04.190611 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:04.192635 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.192635 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.192637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.192847 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.193076 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:04.193077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:04.193077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:04.193135 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:04.193426 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:04.193427 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:04.193486 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 19:10:04.193538 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:10:04.193643 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 19:10:04.194389 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:04.185309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.26.78.252:34305)
0929 19:10:04.185414 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.185416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.185424 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:04.185443 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.185444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.185454 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.185941 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.185973 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.186777 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.187889 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:04.188199 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:04.190697 (+ 2498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:04.190740 (+ 43us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:04.190741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:04.190741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:04.190746 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:04.192513 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.192944 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.192945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.192945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.193001 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:04.193207 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:04.193208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:04.193208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:04.193342 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:04.194180 (+ 838us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:04.194222 (+ 42us) server_negotiation.cc:299] Negotiation successful
0929 19:10:04.194299 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 19:10:04.196161 17264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: Bootstrap starting.
I20260929 19:10:04.196889 17264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:04.197202 17264 log.cc:824] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:04.198140 17264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: No bootstrap required, opened a new log
I20260929 19:10:04.200261 17264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.200397 17264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 773fe074d21e4601b5dd06257ed39b08, State: Initialized, Role: FOLLOWER
I20260929 19:10:04.200476 17264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } }
I20260929 19:10:04.200810 17264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:04.200903 17237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:04.201956 17237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [term 1 FOLLOWER]: Refusing update from remote peer 0c97f3082b23427f92de3213550dcbf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:04.200856 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c97f3082b23427f92de3213550dcbf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "773fe074d21e4601b5dd06257ed39b08"
I20260929 19:10:04.203173 17376 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:04.203043 17238 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c97f3082b23427f92de3213550dcbf3 in term 1.
I20260929 19:10:04.203847 17388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.204676 17388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:10:04.204617 17399 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:04.204819 17399 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:04.206120 17403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c97f3082b23427f92de3213550dcbf3. Latest consensus state: current_term: 1 leader_uuid: "0c97f3082b23427f92de3213550dcbf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.206292 17403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:04.206372 17403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c97f3082b23427f92de3213550dcbf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.206468 17403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:04.223057 17376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c97f3082b23427f92de3213550dcbf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c97f3082b23427f92de3213550dcbf3" member_type: VOTER last_known_addr { host: "127.26.78.254" port: 44831 } } peers { permanent_uuid: "773fe074d21e4601b5dd06257ed39b08" member_type: VOTER last_known_addr { host: "127.26.78.253" port: 41749 } } peers { permanent_uuid: "07308e1c21644539b6b0e5cc63313bb0" member_type: VOTER last_known_addr { host: "127.26.78.252" port: 34305 } } }
I20260929 19:10:04.223191 17376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:04.226634 17380 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:04.227416 17380 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:04.233511 17380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c97f3082b23427f92de3213550dcbf3: Generated new TSK 0
I20260929 19:10:04.233700 17380 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:10:04.257370 17193 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.256020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.26.78.254:44831)
0929 19:10:04.256064 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.256066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.256702 (+ 636us) server_negotiation.cc:407] Connection header received
0929 19:10:04.256805 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.256806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.256818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.256849 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.257291 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41096: BlockingRecv error: recv got EOF from 127.0.0.1:41096 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 19:10:04.258121 17342 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.256421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.26.78.252:34305)
0929 19:10:04.256460 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.256462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.257615 (+ 1153us) server_negotiation.cc:407] Connection header received
0929 19:10:04.257712 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.257714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.257724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.257752 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.258062 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34530: BlockingRecv error: recv got EOF from 127.0.0.1:34530 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 19:10:04.262889 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.262087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.26.78.253:41749)
0929 19:10:04.262131 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.262133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.262264 (+ 131us) server_negotiation.cc:407] Connection header received
0929 19:10:04.262361 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.262362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.262373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.262415 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.262823 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53228: BlockingRecv error: recv got EOF from 127.0.0.1:53228 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 19:10:04.265234 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.264526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.26.78.253:41749)
0929 19:10:04.264567 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.264568 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.264737 (+ 169us) server_negotiation.cc:407] Connection header received
0929 19:10:04.264839 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.264840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.264858 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.264886 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.265188 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53236: BlockingRecv error: recv got EOF from 127.0.0.1:53236 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 19:10:04.268236 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.267518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53248 (local address 127.26.78.253:41749)
0929 19:10:04.267549 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.267551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.267759 (+ 208us) server_negotiation.cc:407] Connection header received
0929 19:10:04.267849 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.267850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.267870 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.267898 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.268190 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53248: BlockingRecv error: recv got EOF from 127.0.0.1:53248 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 19:10:04.272010 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.271298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.26.78.253:41749)
0929 19:10:04.271341 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.271343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.271556 (+ 213us) server_negotiation.cc:407] Connection header received
0929 19:10:04.271647 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.271648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.271657 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.271684 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.271972 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53256: BlockingRecv error: recv got EOF from 127.0.0.1:53256 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260929 19:10:04.277943 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.277111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.26.78.253:41749)
0929 19:10:04.277145 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.277147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.277357 (+ 210us) server_negotiation.cc:407] Connection header received
0929 19:10:04.277481 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.277482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.277493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.277521 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.277895 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53268: BlockingRecv error: recv got EOF from 127.0.0.1:53268 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 19:10:04.288129 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.287120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.26.78.253:41749)
0929 19:10:04.287157 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.287159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.287383 (+ 224us) server_negotiation.cc:407] Connection header received
0929 19:10:04.287649 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.287651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.287663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.287693 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.288077 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53276: BlockingRecv error: recv got EOF from 127.0.0.1:53276 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 19:10:04.306578 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.305569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.26.78.253:41749)
0929 19:10:04.305613 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.305615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.305916 (+ 301us) server_negotiation.cc:407] Connection header received
0929 19:10:04.306025 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.306026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.306037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.306068 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.306521 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53290: BlockingRecv error: recv got EOF from 127.0.0.1:53290 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 19:10:04.340889 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.339981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.26.78.253:41749)
0929 19:10:04.340024 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.340027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.340294 (+ 267us) server_negotiation.cc:407] Connection header received
0929 19:10:04.340411 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.340412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.340425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.340453 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.340828 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53298: BlockingRecv error: recv got EOF from 127.0.0.1:53298 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 19:10:04.407085 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.406343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.26.78.253:41749)
0929 19:10:04.406438 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.406441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.406531 (+ 90us) server_negotiation.cc:407] Connection header received
0929 19:10:04.406607 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.406608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.406624 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.406688 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.407037 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53300: BlockingRecv error: recv got EOF from 127.0.0.1:53300 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260929 19:10:04.537146 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.536389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53308 (local address 127.26.78.253:41749)
0929 19:10:04.536476 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.536479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.536583 (+ 104us) server_negotiation.cc:407] Connection header received
0929 19:10:04.536674 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.536679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.536691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.536750 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.537099 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53308: BlockingRecv error: recv got EOF from 127.0.0.1:53308 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 19:10:04.795298 17268 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:04.794629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.26.78.253:41749)
0929 19:10:04.794701 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:04.794703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:04.794752 (+ 49us) server_negotiation.cc:407] Connection header received
0929 19:10:04.794803 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:04.794804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:04.794815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:04.794914 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:04.795243 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53312: BlockingRecv error: recv got EOF from 127.0.0.1:53312 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 19:10:05.150296 17356 catalog_manager.cc:1259] Loaded cluster ID: b274b90d5ea7478fa66e1f74551bb8a1
I20260929 19:10:05.150377 17356 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: loading cluster ID for follower catalog manager: success
I20260929 19:10:05.151201 17356 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: acquiring CA information for follower catalog manager: success
I20260929 19:10:05.151561 17356 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 07308e1c21644539b6b0e5cc63313bb0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:10:05.205289 17399 catalog_manager.cc:1259] Loaded cluster ID: b274b90d5ea7478fa66e1f74551bb8a1
I20260929 19:10:05.205376 17399 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: loading cluster ID for follower catalog manager: success
I20260929 19:10:05.206342 17399 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: acquiring CA information for follower catalog manager: success
I20260929 19:10:05.206691 17399 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 773fe074d21e4601b5dd06257ed39b08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:10:05.311228 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:05.308417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.26.78.253:41749)
0929 19:10:05.308553 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:05.308555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:05.308642 (+ 87us) server_negotiation.cc:407] Connection header received
0929 19:10:05.308737 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:05.308738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:05.308815 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:05.308901 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:05.309072 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.309590 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.310374 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.310592 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.310677 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:05.310852 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:05.310974 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:05.311055 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:05.311109 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 19:10:05.311130 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":57,"threads_started":1}
I20260929 19:10:05.315168 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:05.312920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.26.78.252:34305)
0929 19:10:05.313047 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:05.313048 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:05.313057 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:05.313115 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:05.313116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:05.313151 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:05.313226 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:05.313302 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.313765 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.314485 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.314678 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.314729 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:05.314848 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:05.314953 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:05.315009 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:05.315043 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 19:10:05.315059 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260929 19:10:05.319669 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:05.316822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.26.78.253:41749)
0929 19:10:05.316859 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:05.316861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:05.316936 (+ 75us) server_negotiation.cc:407] Connection header received
0929 19:10:05.317016 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:05.317018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:05.317030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:05.317085 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:05.317225 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.318021 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.318974 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.319255 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.319315 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:05.319436 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:05.319472 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:05.319516 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:05.319574 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 19:10:05.319597 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:10:05.319703 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:05.316909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.26.78.252:34305)
0929 19:10:05.316947 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:05.316948 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:05.316957 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:05.317047 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:05.317049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:05.317056 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:05.317120 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:05.317260 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.317903 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.318993 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.319183 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.319255 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:05.319385 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:05.319421 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:05.319475 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:05.319612 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 19:10:05.319636 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:10:05.319952 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:05.316737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.26.78.254:44831)
0929 19:10:05.316921 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:05.316923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:05.317158 (+ 235us) server_negotiation.cc:407] Connection header received
0929 19:10:05.317216 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:05.317217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:05.317265 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:05.317344 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:05.317435 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.318167 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.318882 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:05.319183 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:05.319243 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:05.319482 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:05.319591 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:05.319650 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:05.319845 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 19:10:05.319870 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":50,"threads_started":1}
I20260929 19:10:05.327553 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 17121
I20260929 19:10:05.332849 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 17196
I20260929 19:10:05.337720 26939 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZMrn7N/build/release/bin/kudu with pid 17271
2026-09-29T19:10:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2797 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5622 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (295138 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS